:W&BᾊY8’M r7y7t8hnª; 9 /0.29.0+7872fd9d9f86ca5c50617e19cb168122b7f959480.70.0j>ºú‚" 6e65cqiarhos r7y7t8hnŠÖ r7y7t8hnjuju11-uc-san-diegoSPAtop"™ _wandb‚{}  batch_size‚512  dataset_limit‚0.2  dropout‚0.09397600617167448 epochs‚10  hidden_dim‚64 ' learning_rate‚0.0017185915604411852  num_attention_heads‚8  num_branch_encoder_layers‚5  num_embedding_layers‚14  num_encoder_layers‚2Bjjtttt_vanilla_r7y7t8hnb88oquqqwj)jj-spatop-sweepagent-tttt-vanilla-2-spfhcŠ ûÎâÔøê íš3 (H¨¸hp"3.12.10*0.29.0j linux-x86_64:¤£V7 r7y7t8hn&  _runtime‚0  _wandb.runtime‚0o¶÷ jÒä -Linux-6.8.0-136-generic-x86_64-with-glibc2.36CPython 3.12.10 úÎâÔø‰¼©*-of*=/data/spatop/options_files/sweep_tttt_jj/config_r7y7t8hn.json*-l* /scratch/logs*-n*jjtttt_vanilla_r7y7t8hn2-m spanet.trainRjacksonjulian2014@gmail.comZ /scratch/logsb)jj-spatop-sweepagent-tttt-vanilla-2-spfhcr/opt/conda/bin/pythonº 5swqra8as586ffb1jt0bl8zxbd536tuz/ð pÉÒÀ€ˆ€’NVIDIA A100-SXM4-80GB˜¢ /€€€¥È.€ ÔªÉ ª€àäÓ½ÂS NVIDIA A100-SXM4-80GB€€€€À€6"Ampere*(GPU-1512559d-7724-700d-39a3-dab660c9a52eÊ13.3º 5swqra8as586ffb1jt0bl8zxbd536tuzµª½Ç2  requirements.txt,_áW r7y7t8hnZF (H¨¸ (H¨¸ (hpð"3.12.10*0.29.0j linux-x86_64UØÍrY r7y7t8hnZH (H¨¸ (H¨¸ (8hpð"3.12.10*0.29.0j linux-x86_64ÁWÓâ*  r7y7t8hnb trainer/global_step2ÉðwE\  r7y7t8hnZK (H¨¸ (H¨¸(8hpð"3.12.10*0.29.0j linux-x86_64‰îÓ/  r7y7t8hnb*"trainer/global_step2ô•_K  r7y7t8hnj: üÎâÔ›·š*LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] söëF“  r7y7t8hnj  üÎâÔ Û΃î â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┓ ┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃ ┡â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”┩ │ 0 │ embedding │ MultiInputVectorEmbed… │ 220 K │ train │ 0 │ │ 1 │ encoder │ JetEncoder │ 166 K │ train │ 0 │ │ 2 │ branch_decoders │ ModuleList │ 11.6 M │ train │ 0 │ │ 3 │ regression_decoder │ RegressionDecoder │ 0 │ train │ 0 │ │ 4 │ classification_decoder │ ClassificationDecoder │ 0 │ train │ 0 │ │ │ other params │ n/a │ 48 │ n/a │ n/a │ └───┴────────────────────────┴────────────────────────┴────────┴───────┴───────┘ ˆEæ\ r7y7t8hnjÊ üÎâÔ®À„·Trainable params: 12.0 M Non-trainable params: 86 Total params: 12.0 M Total estimated model params size (MB): 47.847 Modules in train mode: 5033 Modules in eval mode: 0 Total FLOPs: 0 P—½ðq r7y7t8hnj` üÎâÔè„à„Pwandb: WARNING Config item 'hidden_dim' was locked by 'sweep' (ignored update). eÇØ{ r7y7t8hnjj üÎâÔÀ‹ð„Zwandb: WARNING Config item 'num_embedding_layers' was locked by 'sweep' (ignored update). ßÌg¨y r7y7t8hnjh üÎâÔÈ—ù„Xwandb: WARNING Config item 'num_encoder_layers' was locked by 'sweep' (ignored update). ÍÍF€ r7y7t8hnjo üÎâÔ€‚ˆ…_wandb: WARNING Config item 'num_branch_encoder_layers' was locked by 'sweep' (ignored update). Ùui¹z r7y7t8hnji üÎâÔø³”…Ywandb: WARNING Config item 'num_attention_heads' was locked by 'sweep' (ignored update). ¥nYót r7y7t8hnjc üÎâÔˆÒ¤…Swandb: WARNING Config item 'dataset_limit' was locked by 'sweep' (ignored update). pÞOØq r7y7t8hnj` üÎâÔ¨Ù¬…Pwandb: WARNING Config item 'batch_size' was locked by 'sweep' (ignored update). ò¤É¾t r7y7t8hnjc üÎâÔ  ²…Swandb: WARNING Config item 'learning_rate' was locked by 'sweep' (ignored update). Æ‚n r7y7t8hnj] üÎâÔð ·…Mwandb: WARNING Config item 'dropout' was locked by 'sweep' (ignored update). :cˆ-m r7y7t8hnj\ üÎâÔˆ‡½…Lwandb: WARNING Config item 'epochs' was locked by 'sweep' (ignored update). Ñ®)ý_ r7y7t8hn*Í  transformer_dim‚64  transformer_dim_scale‚2.0  initial_embedding_dim‚64  position_embedding_dim‚64 ! num_branch_embedding_layers‚3  num_jet_embedding_layers‚0  num_jet_encoder_layers‚2  num_detector_layers‚2  num_regression_layers‚3  num_classification_layers‚3  split_symmetric_attention‚1 ! transformer_activation‚"gelu"  skip_connections‚1 ( "initial_embedding_skip_connections‚1  linear_block_type‚"GRU"  transformer_type‚"Gated"  linear_activation‚"gelu"  normalization‚ "LayerNorm"  masking‚ "Filling" linear_prelu_activation‚true  use_pairwise_interactions‚0  num_pairwise_features‚4  pairwise_embedding_dim‚8  pairwise_input_source‚""  use_moe‚false  num_experts‚8  num_experts_per_tok‚2  moe_loss_scale‚0.01 F event_info_file‚2"/data/spatop/event_files/tttt/tttt_hadronic.yaml" = training_file‚+"/data/spatop/tttt_15M/h5/tttt_training.h5"  validation_file‚""  testing_file‚""  normalize_features‚1  limit_to_num_jets‚0  balance_particles‚0  balance_jets‚0 ! balance_classifications‚false  partial_events‚1  dataset_randomization‚0  train_validation_split‚0.95  limit_val_batches‚1.0  num_dataloader_workers‚0  mask_sequence_vectors‚1  combine_pair_loss‚"min"  optimizer‚"AdamW"  focal_gamma‚0.0  combinatorial_scale‚0.0 # learning_rate_warmup_epochs‚1.0  learning_rate_cycles‚1  assignment_loss_scale‚1.0  detection_loss_scale‚1.0  kl_loss_scale‚0.0  regression_loss_scale‚0.0 ! classification_loss_scale‚0.0 ! use_mass_classification‚false  mass_classes‚[]  balance_losses‚true  l2_penalty‚0.0002  gradient_clip‚10.0 num_gpu‚1  verbose_output‚false  usable_gpus‚""  trial_time‚"" $ trial_output_dir‚"./test_output"ñçÚë r7y7t8hnjÙ üÎâÔð‡ÚŠÈ/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. ‡õN r7y7t8hnj¼ üÎâÔЯ슫/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. x ë r7y7t8hnjÙ ÿÎâÔ Ö¨¬È/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. ÒÚÏFP r7y7t8hnj¾ ÿÎâÔÈò¶¬­/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. óLT2‚(:† ‹ÏâÔ®±¯u gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.177' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6354 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0`‹Š¶û‚(:ñ ‹ÏâÔÅ÷¬{ disk.rbd2.out‚1.09375 disk.rbd3.out‚0 disk.rbd0.out‚0( proc.memory.availableMB‚ 3121.7578125 disk.sda3.out‚ 0.55078125 disk.dm-1.out‚ 0.55078125) disk./.usagePercent‚69.98398293982177 disk.sda2.in‚0 disk.nvme0n1.in‚ 0.00390625 proc.cpu.threads‚17 disk.dm-1.in‚0 disk.sda.out‚ 0.55078125 disk.sda1.out‚0 disk.nvme0c0n1.in‚ 0.00390625% disk./.usageGB‚1042.8119201660156 disk.dm-0.out‚124.2158203125 disk.rbd1.out‚0 disk.drbd1777.out‚0 cpu‚8.811560553106979 network.recv‚43346 disk.rbd0.in‚0* proc.memory.percent‚17.098379135131836 disk.drbd1777.in‚0 disk.sda3.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 0.00390625 disk.rbd2.in‚0# disk.nvme0n1.out‚124.2158203125 disk.sda1.in‚0% disk.nvme0c0n1.out‚124.2158203125 disk.rbd1.in‚0 disk.sda.in‚0$ memory_percent‚80.94630241394043" proc.memory.rssMB‚ 2801.3984375 network.sent‚34601 disk.rbd3.in‚01£û] ‚(:… šÏâÔÂŵu gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚95.485' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.097 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0cÂûû!‚(:ñ šÏâÔ±’„x% disk./.usageGB‚1042.3333892822266 disk.sda3.out‚ 1.07421875 disk.dm-1.out‚ 1.07421875 disk.rbd2.out‚1.125# disk.nvme0n1.out‚340.8017578125 disk.sda2.out‚0 network.recv‚47896 disk.dm-1.in‚0 disk.rbd2.in‚0 disk.nvme0c0n1.in‚ 0.00390625 cpu‚10.695284684740132) disk./.usagePercent‚69.95186833069653 disk.sda3.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚0 disk.sda.in‚0 disk.sda1.out‚0 disk.drbd1777.out‚0) proc.memory.availableMB‚ 3121.33984375 disk.rbd3.out‚0 disk.dm-0.out‚340.8017578125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2801.4140625* proc.memory.percent‚17.098474502563477 network.sent‚44392 disk.rbd3.in‚0 disk.nvme0n1.in‚ 0.00390625 proc.cpu.threads‚17 disk.drbd1777.in‚0% disk.nvme0c0n1.out‚340.8017578125$ memory_percent‚80.94885349273682 disk.sda.out‚ 1.07421875 disk.rbd0.out‚0 disk.dm-0.in‚ 0.00390625—;Äa3"b/ lr-AdamW/pg1"trainer/global_step2:JP<Ñsˆ3#b/ lr-AdamW/pg2"trainer/global_step2:JP¼º„-$) _wandbruntime‚44  _runtime‚44s)Ê/Â%½ ' lr-AdamW/pg1‚0.00013942216301592397 ' lr-AdamW/pg2‚0.00013942216301592397 trainer/global_step‚49 % _timestamp‚1.7883892891233017e+09 _runtime‚ 44.879350473 _step‚0/¼yÄ&¿ ' lr-AdamW/pg2‚0.00013942216301592397 trainer/global_step‚49 % _timestamp‚1.7883892891233017e+09 _runtime‚ 44.879350473 _step‚0 ' lr-AdamW/pg1‚0.00013942216301592397¾N•1i'be BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated"trainer/global_step2:JP‹ÍÊ-p(bl IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated"trainer/global_step2:JP¾¤Y/j)bf CDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current"trainer/global_step2:JP‹D@¡q*bm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current"trainer/global_step2:JPn÷Tj+bf CDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current"trainer/global_step2:JP(i,be BDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak"trainer/global_step2:JPg·Õ4j-bf CDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated"trainer/global_step2:JPÖt*g.bc @DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed"trainer/global_step2:JP}ÿdk/bg DDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current"trainer/global_step2:JP„Wmk0bg DDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak"trainer/global_step2:JP¡zÂc1b_ bb ?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed"trainer/global_step2:JPeH’o?bk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed"trainer/global_step2:JPWÖÉùl@bh EDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current"trainer/global_step2:JPû.Gø^AbZ 7DeviceStatsMonitor.on_train_batch_start/num_device_free"trainer/global_step2:JPRWÆÏnBbj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak"trainer/global_step2:JP>øуhCbd ADeviceStatsMonitor.on_train_batch_start/active.large_pool.current"trainer/global_step2:JPkÕ§üdDb` =DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak"trainer/global_step2:JPïÛ"#sEbo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak"trainer/global_step2:JPÏÞòpFbl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current"trainer/global_step2:JPO¼rgiGbe BDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current"trainer/global_step2:JPL³õdHb` =DeviceStatsMonitor.on_train_batch_start/segment.all.allocated"trainer/global_step2:JPYËÀwlIbh EDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed"trainer/global_step2:JP ¸¦uiJbe BDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed"trainer/global_step2:JPÀíó™eKba >DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak"trainer/global_step2:JP&•WkLbg DDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak"trainer/global_step2:JPÁî£ylMbh EDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current"trainer/global_step2:JPü™$qNbm JDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current"trainer/global_step2:JP´SªkObg DDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated"trainer/global_step2:JP¤›ePba >DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed"trainer/global_step2:JPª7^únQbj GDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated"trainer/global_step2:JP:%apRbl IDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current"trainer/global_step2:JP{Æ€tSbp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed"trainer/global_step2:JPgýŠ%fTbb ?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak"trainer/global_step2:JP$__gUbc @DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed"trainer/global_step2:JPF€Ð3cVb_ DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak"trainer/global_step2:JP†Ëêenba >DeviceStatsMonitor.on_train_batch_start/allocation.all.current"trainer/global_step2:JP‹Óïvobr ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current"trainer/global_step2:JP”¥°Ërpbn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated"trainer/global_step2:JP %nqbj GDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current"trainer/global_step2:JPn“¿Wfrbb ?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak"trainer/global_step2:JPÅö>hsbd ADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed"trainer/global_step2:JP`îØpntbj GDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated"trainer/global_step2:JPnZ.fubb ?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak"trainer/global_step2:JPXែvvbr ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current"trainer/global_step2:JP$ªU`ewba >DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak"trainer/global_step2:JP\í—!jxbf CDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated"trainer/global_step2:JPYknybj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak"trainer/global_step2:JPÃ- ™]zbY 6DeviceStatsMonitor.on_train_batch_start/max_split_size"trainer/global_step2:JP´^ðŒr{bn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated"trainer/global_step2:JPÍiáœs|bo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated"trainer/global_step2:JPÀTJl}bh EDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak"trainer/global_step2:JPF(YÑl~bh EDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated"trainer/global_step2:JPôêhbd ADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed"trainer/global_step2:JPÅ.^–p€bk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed"trainer/global_step2:JPÍ7¸™`b[ 8DeviceStatsMonitor.on_train_batch_start/segment.all.peak"trainer/global_step2:JPGu}-g‚bb ?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed"trainer/global_step2:JP+–RQtƒbo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated"trainer/global_step2:JP“°;by„bt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JPq¿2c…b^ ;DeviceStatsMonitor.on_train_batch_start/segment.all.current"trainer/global_step2:JPέXi†bd ADeviceStatsMonitor.on_train_batch_start/active.small_pool.current"trainer/global_step2:JP‰9äj‡be BDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak"trainer/global_step2:JP¡¤Ïoˆbj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak"trainer/global_step2:JPÒœ0¢l‰bg DDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated"trainer/global_step2:JPw[æoŠbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed"trainer/global_step2:JP ï²`‹b[ 8DeviceStatsMonitor.on_train_batch_start/active.all.freed"trainer/global_step2:JPguWñrŒbm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current"trainer/global_step2:JPïú³\qbl IDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current"trainer/global_step2:JP(Ñ»ûsŽbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated"trainer/global_step2:JPÆß(­ybt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JPø0Èókbf CDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated"trainer/global_step2:JP˜¥÷“g‘bb ?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed"trainer/global_step2:JP)$Rd’b_ DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 3316844640256 RGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚834966 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚516 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 3302594467328 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 _runtime‚ 44.882945874 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 3316844640256 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚1061750 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1127 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242788352 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 trainer/global_step‚49 _step‚1 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚834966 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497871360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 40328510464 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 3240992120502 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 3240783090926 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 39907319022 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1040446 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 3276728685568 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 41949487616 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 LBDeÒÿ3 viceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26032 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚1875410 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚260505 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 3302619371008 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 3316844640256 RGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚834966 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚516 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 3302594467328 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 _runtime‚ 44.882945874 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 3316844640256 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚1061750 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1127 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242788352 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 trainer/global_step‚49 _step‚1 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚834966 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497871360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 40328510464 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 3240992120502 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 3240783090926 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 39907319022 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1040446 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 3276728685568 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 41949487616 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26032 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507#û^î’¥‚(:‡ ©ÏâÔÛáÊu gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚123.366' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.6732 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0# êù¦‚(:î ©ÏâÔ³ÝÙw disk.sda3.out‚1.734375$ memory_percent‚80.95500469207764 disk.sda.in‚0 disk.drbd1777.in‚0& disk.nvme0c0n1.out‚341.94091796875 proc.cpu.threads‚17 disk.sda1.in‚0 disk.dm-1.in‚0 disk.nvme0n1.in‚ 0.00390625) proc.memory.availableMB‚ 3120.33203125) proc.memory.percent‚17.09880828857422( disk./.usagePercent‚69.9518711467794 disk.sda.out‚1.734375 disk.nvme0c0n1.in‚ 0.00390625 disk.rbd3.out‚0 disk.sda3.in‚0 disk.rbd1.out‚0$ disk.nvme0n1.out‚341.94091796875 network.sent‚50167 disk.sda1.out‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 0.00390625 disk.rbd0.out‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.rbd1.in‚0 disk.rbd2.out‚ 2.3515625 proc.memory.rssMB‚ 2801.46875 network.recv‚51842 disk.rbd0.in‚0 disk.sda2.in‚0! disk.dm-0.out‚341.94091796875 disk.rbd2.in‚0 cpu‚12.197003091012522% disk./.usageGB‚1042.3334312438965 disk.dm-1.out‚1.734375 Ö#e§b` =DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak"trainer/global_step2:JP@3/Ý]¨bX 5DeviceStatsMonitor.on_train_batch_end/active.all.peak"trainer/global_step2:JP`Ù¿ãr©bm JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated"trainer/global_step2:JPnCrÊmªbh EDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated"trainer/global_step2:JP^¤^Îe«b` =DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak"trainer/global_step2:JP’Ks'a¬b\ 9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak"trainer/global_step2:JPPe"Ði­bd ADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed"trainer/global_step2:JPw‚öòV®bQ .DeviceStatsMonitor.on_train_batch_end/num_ooms"trainer/global_step2:JPg‚Lan¯bi FDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed"trainer/global_step2:JPrˆ6aj°be BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated"trainer/global_step2:JP7Ü‚h±bc @DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current"trainer/global_step2:JPÍüpi²bd ADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated"trainer/global_step2:JPFÈác³b^ ;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak"trainer/global_step2:JP\²zk´bf CDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed"trainer/global_step2:JP͘dklµbg DDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed"trainer/global_step2:JP;‹úÆb¶b] :DeviceStatsMonitor.on_train_batch_end/active.all.allocated"trainer/global_step2:JPô;hn·bi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed"trainer/global_step2:JPœ¨Ñ¯m¸bh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak"trainer/global_step2:JP¥’AÜg¹bb ?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current"trainer/global_step2:JP»Þ·¿fºba >DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed"trainer/global_step2:JPrã–iq»bl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated"trainer/global_step2:JP«wŸp¼bk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current"trainer/global_step2:JPU“Äj½be BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated"trainer/global_step2:JP»Xlg¾bb ?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current"trainer/global_step2:JP­¡Ïm¿bh EDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current"trainer/global_step2:JP¼³‰kÀbf CDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current"trainer/global_step2:JP/ÑkÁbf CDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak"trainer/global_step2:JPì¼ù=bÂb] :DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams"trainer/global_step2:JP³O‰eÃb` =DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed"trainer/global_step2:JPWüSkÄbf CDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current"trainer/global_step2:JPŸš^ïhÅbc @DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current"trainer/global_step2:JPS0¨iÆbd ADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated"trainer/global_step2:JPy9JÒ\ÇbW 4DeviceStatsMonitor.on_train_batch_end/max_split_size"trainer/global_step2:JPgü jgÈbb ?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed"trainer/global_step2:JPWFlmmÉbh EDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak"trainer/global_step2:JPÕ=ÅmÊbh EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed"trainer/global_step2:JP-[1$iËbd ADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current"trainer/global_step2:JP\Ç¿¦oÌbj GDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current"trainer/global_step2:JP$DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current"trainer/global_step2:JPÌ(j½fÓba >DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed"trainer/global_step2:JPkmåËwÔbr ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JP?¯¼lÕbg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak"trainer/global_step2:JPêp7qÖbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated"trainer/global_step2:JPË^T^×bY 6DeviceStatsMonitor.on_train_batch_end/num_device_alloc"trainer/global_step2:JPv ¢ŸnØbi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed"trainer/global_step2:JPT$bKfÙba >DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated"trainer/global_step2:JP:ÅêíqÚbl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated"trainer/global_step2:JP ÄÍhÛbc @DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated"trainer/global_step2:JPF.ÆjÜbe BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated"trainer/global_step2:JPaÑèrÝbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak"trainer/global_step2:JP[%¢ˆrÞbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated"trainer/global_step2:JPð7Å¿aßb\ 9DeviceStatsMonitor.on_train_batch_end/segment.all.current"trainer/global_step2:JP¤@;„ràbm JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated"trainer/global_step2:JPŒ™31nábi FDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed"trainer/global_step2:JPdzPGbâb] :DeviceStatsMonitor.on_train_batch_end/allocation.all.freed"trainer/global_step2:JP6ÜÓlãbg DDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak"trainer/global_step2:JPß@Äeäb` =DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed"trainer/global_step2:JP^Â¥ fåba >DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed"trainer/global_step2:JPþ¯Fémæbh EDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated"trainer/global_step2:JPÛ•Ž§mçbh EDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed"trainer/global_step2:JPœ2ÈŠlèbg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak"trainer/global_step2:JPŠçcéb^ ;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated"trainer/global_step2:JPëããidêb_ DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak"trainer/global_step2:JPIr´pìbk HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated"trainer/global_step2:JP‘¹Ohjíbe BDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current"trainer/global_step2:JPcêŽ×mîbh EDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak"trainer/global_step2:JP(8hoïbj GDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current"trainer/global_step2:JP{tHmðbh EDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current"trainer/global_step2:JP ý%gñbb ?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed"trainer/global_step2:JP5pooòbj GDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current"trainer/global_step2:JP6ð“lóbg DDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak"trainer/global_step2:JPÿ¶dooôbj GDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated"trainer/global_step2:JPÝÎ jõbe BDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak"trainer/global_step2:JPm·Åhöbc @DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak"trainer/global_step2:JP”ô¤—h÷bc @DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current"trainer/global_step2:JPýI÷Œdøb_ ‘ÔõqŒbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated"trainer/global_step2:JPáW:knbi FDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current"trainer/global_step2:JPu+ÑwŽbr ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JPb(Ylbg DDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated"trainer/global_step2:JP‚b¾^fba >DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak"trainer/global_step2:JP*`Ñdd‘b_ DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed"trainer/global_step2:JP»æºk›bf CDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed"trainer/global_step2:JP~aóhœbc @DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak"trainer/global_step2:JPÜeb` =DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak"trainer/global_step2:JP¥ÐÖržbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak"trainer/global_step2:JPþhŸbc @DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed"trainer/global_step2:JP‹ß¹šg bb ?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak"trainer/global_step2:JP3‰0;e¡b` =DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak"trainer/global_step2:JP Áû.¢) _wandbruntime‚45  _runtime‚45~ê¾ç£–M GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚265730 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 276067328 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 42779470848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 trainer/global_step‚49 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 40931384832 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1127 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 41126914560 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚1082648 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 % _timestamp‚1.7883892898746421e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1061318 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 40701201970 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚851486 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 3341750683136 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 3410648270848 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚1934134 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 3341750683136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1061318 MCDeviceS{­ûµtatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚1082648 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 3305176358450 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚552 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚851484 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚784552 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 3368144867328 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚519377 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 40931384832 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1934134 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 3382894637056 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚785107 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 3264492195840 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 3341767722496 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 3264475156480 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1912802 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1109 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚851486 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚519374 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 NDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 3410924338176 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 3382682067968 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 41126914560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚555 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 3382894637056G>— &¤šM NDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 3410924338176 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 3382682067968 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 41126914560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚555 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 3382894637056 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚265730 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 276067328 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 42779470848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 trainer/global_step‚49 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 40931384832 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1127 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 41126914560 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚1082648 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 % _timestamp‚1.7883892898746421e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1061318 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 40701201970 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚851486 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 3341750683136 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 3410648270848 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚1934134 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 3341750683136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1061318 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚1082648 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 3305176358450 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚552 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚851484 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚784552 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 3368144867328 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚519377 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 40931384832 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1934134 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 3382894637056 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚785107 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 3264492195840 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 3341767722496 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 3264475156480 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1912802 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1109 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚851486 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚519374 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750qE*A¥b< loss/FRt1/assignment_loss"trainer/global_step2:JP,A¦b< loss/FRt3/assignment_loss"trainer/global_step2:JPŸ&9C§b> loss/SRqqt1/assignment_loss"trainer/global_step2:JPkG¾ A¨b< loss/FBt3/assignment_loss"trainer/global_step2:JPnaNØB©b= loss/SRqqt4/detection_loss"trainer/global_step2:JPU[ÐåAªb< loss/FBt4/assignment_loss"trainer/global_step2:JP¤krt@«b; loss/FRt1/detection_loss"trainer/global_step2:JP_ÎêˆA¬b< loss/FBt1/assignment_loss"trainer/global_step2:JP±Š“@­b; loss/FRt4/detection_loss"trainer/global_step2:JPó`©0A®b< loss/FRt2/assignment_loss"trainer/global_step2:JPqÒÈC¯b> loss/SRqqt3/assignment_loss"trainer/global_step2:JPû\zrB°b= loss/SRqqt2/detection_loss"trainer/global_step2:JP·*i@±b; loss/FBt2/detection_loss"trainer/global_step2:JPéÌW-²b( epoch"trainer/global_step2:JPf$|A³b< loss/FRt4/assignment_loss"trainer/global_step2:JPš9»ñC´b> loss/SRqqt2/assignment_loss"trainer/global_step2:JP«BlAµb< loss/FBt2/assignment_loss"trainer/global_step2:JP9@¶b; loss/FRt2/detection_loss"trainer/global_step2:JPÖ|fË@·b; loss/FRt3/detection_loss"trainer/global_step2:JP9©—äB¸b= loss/SRqqt1/detection_loss"trainer/global_step2:JP|˜*@¹b; loss/FBt4/detection_loss"trainer/global_step2:JPÔ¬õ7ºb2 loss/total_loss"trainer/global_step2:JPFÌ[C»b> loss/SRqqt4/assignment_loss"trainer/global_step2:JP°lOB¼b= loss/SRqqt3/detection_loss"trainer/global_step2:JPѬ=R@½b; loss/FBt1/detection_loss"trainer/global_step2:JPÒK)f@¾b; loss/FBt3/detection_loss"trainer/global_step2:JPT{¶ž.¿) _wandbruntime‚45  _runtime‚45™)£AJÀÄ .loss/FRt1/detection_loss‚2.676323097574861 /loss/FBt1/assignment_loss‚2.363439955030169 .loss/FRt4/detection_loss‚6.091266248930532 trainer/global_step‚49 /loss/FRt2/assignment_loss‚7.394009846512989 _runtime‚ 45.632343386 1loss/SRqqt3/assignment_loss‚5.524694427465781 1loss/SRqqt2/detection_loss‚3.4881199423712914 /loss/FBt2/detection_loss‚18.267784658405517 epoch‚0 $ _timestamp‚1.788389289876522e+09 /loss/FRt4/assignment_loss‚7.225422847838629 1loss/SRqqt2/assignment_loss‚5.447228253489793 0loss/FBt2/assignment_loss‚2.3459028005599976 /loss/FRt2/detection_loss‚3.8529073861978387 /loss/FRt3/detection_loss‚3.2849765978753567 0loss/SRqqt1/detection_loss‚2.748243946675994 /loss/FBt4/detection_loss‚1.6872700571496453 &loss/total_loss‚135.88270949380313 _step‚3 2loss/SRqqt4/assignment_loss‚5.3841968704672425 1loss/SRqqt3/detection_loss‚1.7262033492278976 .loss/FBt1/detection_loss‚9.605291274615697 .loss/FBt3/detection_loss‚16.25288529396057 /loss/FRt1/assignment_loss‚7.207538688884062 /loss/FRt3/assignment_loss‚7.439536437988282 1loss/SRqqt1/assignment_loss‚5.359752274360977 /loss/FBt3/assignment_loss‚2.177973967790604 1loss/SRqqt4/detection_loss‚6.4291125378187965 /loss/FBt4/assignment_loss‚1.902628732610632XêÛNÁÈ /loss/FRt1/assignment_loss‚7.207538688884062 /loss/FRt3/assignment_loss‚7.439536437988282 1loss/SRqqt1/assignment_loss‚5.359752274360977 /loss/FBt3/assignment_loss‚2.177973967790604 1loss/SRqqt4/detection_loss‚6.4291125378187965 /loss/FBt4/assignment_loss‚1.902628732610632 .loss/FRt1/detection_loss‚2.676323097574861 /loss/FBt1/assignment_loss‚2.363439955030169 .loss/FRt4/detection_loss‚6.091266248930532 trainer/global_step‚49 /loss/FRt2/assignment_loss‚7.394009846512989 _runtime‚ 45.632343386 1loss/SRqqt3/assignment_loss‚5.524694427465781 1loss/SRqqt2/detection_loss‚3.4881199423712914 /loss/FBt2/detection_loss‚18.267784658405517 epoch‚0 $ _timestamp‚1.788389289876522e+09 /loss/FRt4/assignment_loss‚7.225422847838629 1loss/SRqqt2/assignment_loss‚5.447228253489793 0loss/FBt2/assignment_loss‚2.3459028005599976 /loss/FRt2/detection_loss‚3.8529073861978387 /loss/FRt3/detection_loss‚3.2849765978753567 0loss/SRqqt1/detection_loss‚2.748243946675994 /loss/FBt4/detection_loss‚1.6872700571496453 &loss/total_loss‚135.88270949380313 _step‚3 2loss/SRqqt4/assignment_loss‚5.3841968704672425 1loss/SRqqt3/detection_loss‚1.7262033492278976 .loss/FBt1/detection_loss‚9.605291274615697 .loss/FBt3/detection_loss‚16.25288529396057©›hFœÂ‚(:‘ ¸ÏâÔÒ–óu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚136.784' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.356799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ØÛÒXÂ(:ý ¸ÏâÔï¥Êz disk.drbd1777.out‚0! disk.dm-0.out‚440.41845703125% disk./.usageGB‚1042.3331298828125$ disk.nvme0n1.out‚440.41845703125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd1.out‚0& disk.nvme0c0n1.out‚440.41845703125 disk.rbd3.out‚ 0.12109375 disk.sda3.in‚0 disk.sda3.out‚2.046875 disk.rbd2.out‚ 3.4697265625 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-1.out‚2.046875) proc.memory.availableMB‚ 3116.95703125) proc.memory.percent‚17.09883213043213 network.sent‚97827 network.recv‚86154 disk.sda2.out‚0 disk.dm-1.in‚0$ memory_percent‚80.97560405731201 cpu‚13.425636271315584 disk.sda1.out‚0 disk.rbd0.out‚0 disk.nvme0c0n1.in‚ 1.83203125 disk.dm-0.in‚ 1.83203125# proc.memory.rssMB‚ 2801.47265625 proc.cpu.threads‚17( disk./.usagePercent‚69.9518509221842 disk.sda.in‚0 disk.rbd1.in‚0 disk.nvme0n1.in‚ 1.83203125 disk.drbd1777.in‚0 disk.rbd2.in‚0 disk.sda.out‚2.046875>g¡œÄ‚(:‘ ÇÏâÔÜ·°u gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚150.496' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.099200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XwœåüÅ‚(:ñ ÇÏâÔæ½ûx disk.rbd0.out‚0 disk.sda2.out‚0) disk./.usagePercent‚69.95191594809789 disk.sda3.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚2.625 disk.rbd2.in‚0# proc.memory.rssMB‚ 2801.48828125 network.sent‚137713 disk.drbd1777.out‚0 disk.sda2.in‚0# memory_percent‚80.9729814529419) proc.memory.availableMB‚ 3117.38671875 disk.sda.in‚0% disk.nvme0c0n1.out‚539.6044921875 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚ 1.83203125 disk.nvme0n1.in‚ 1.83203125# disk.nvme0n1.out‚539.6044921875 network.recv‚92342 cpu‚14.449484057275454 disk.sda3.out‚2.625 disk.sda.out‚2.625 disk.sda1.in‚0 disk.rbd3.in‚0 proc.cpu.threads‚17 disk.rbd2.out‚ 4.5439453125$ disk./.usageGB‚1042.334098815918 disk.rbd1.out‚0 disk.dm-0.in‚ 1.83203125 disk.dm-0.out‚539.6044921875) proc.memory.percent‚17.09892749786377 disk.rbd1.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd3.out‚ 0.12109375æ¹R³.Æ) _wandbruntime‚86  _runtime‚86}¦Ÿ¿Ç¹ % lr-AdamW/pg1‚0.000281689676297479 % lr-AdamW/pg2‚0.000281689676297479 trainer/global_step‚99 % _timestamp‚1.7883893306501038e+09 _runtime‚ 86.406091598 _step‚4Cf_\ÃȽ % lr-AdamW/pg1‚0.000281689676297479 % lr-AdamW/pg2‚0.000281689676297479 trainer/global_step‚99 % _timestamp‚1.7883893306501038e+09 _runtime‚ 86.406091598 _step‚4'=´s.É) _wandbruntime‚86  _runtime‚86 ´o«3$Ê”O @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚3748362 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚504 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 6662302813184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246982656 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _runtime‚ 86.411024317 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2087398 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2087398 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 6608743854080 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1130 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1660964 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚518766 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 6608743854080 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 6578869283328 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 6460682668710 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 83408626176 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 83161643520 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1011040 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 79820628646 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 _step‚5 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 6608956409856 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 6460473639134 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3769668 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 % _timestamp‚1.7883893306511538e+09 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 6608956409856 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 6528680531968 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1530310 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 6528663492608 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1660966 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1660964 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚501 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 6528663492608 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 80275877888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1529806 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1660966 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2108702 trainer/global_step‚99 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 80080361472 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 6528680531968 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 79628638430 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 OEDeviceS¬åA0gtatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 6578894187008 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 80275877888 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚519267 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 6662030926848 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 80080361472 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 6608743854080 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1130 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1660964 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚518766 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 6608743854080 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 6578869283328 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 6460682668710 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 83408626176 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 83161643520 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1011040 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 79820628646 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 _step‚5 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 6608956409856 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 6460473639134 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3769668 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 % _timestamp‚1.7883893306511538e+09 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 6608956409856 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 6528680531968 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1530310 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 6528663492608 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1660966 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1660964 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚501 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 6528663492608 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 80275877888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1529806 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1660966 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2108702 trainer/global_step‚99 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 80080361472 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 6528680531968 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 79628638430 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 6578894187008 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 80275877888 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚519267ï8¼«.Ì) _wandbruntime‚87  _runtime‚87Žœ¤&ÍžM H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 6644419683328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 6674793850880 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚523793 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 6525075936442 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 _runtime‚ 87.163257268 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 6644394779648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3807086 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2108270 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 81074281984 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 80878752256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3785754 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1677486 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 6593702529536 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 81074281984 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1677484 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2108270 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚3807086 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 6674793850880 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 trainer/global_step‚99 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 NDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1544867 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚6 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1112 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 6444461424640 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 6444444385280 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 80614511802 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 6593719568896 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 80878752256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888Ù…¶¨&΢M VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1112 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 6444461424640 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 6444444385280 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 80614511802 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 6593719568896 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 80878752256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 6644419683328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 6674793850880 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚523793 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 6525075936442 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 _runtime‚ 87.163257268 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 6644394779648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3807086 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2108270 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 81074281984 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 80878752256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3785754 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1677486 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 6593702529536 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 81074281984 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1677484 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2108270 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚3807086 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 6674793850880 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 trainer/global_step‚99 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 NDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1544867 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚6zFw”.Ï) _wandbruntime‚87  _runtime‚87¾f½IÐà 2loss/SRqqt2/assignment_loss‚3.8212107335678254 1loss/FBt3/assignment_loss‚0.46430827758740634 1loss/SRqqt3/detection_loss‚1.3764205417446074 _step‚7 /loss/FRt3/assignment_loss‚7.406675979655276 1loss/SRqqt1/assignment_loss‚3.347191764248742 .loss/FRt2/detection_loss‚2.966327424504255 .loss/FBt1/detection_loss‚6.029088598939608 +loss/FRt1/assignment_loss‚ 7.14851705983 /loss/FRt4/assignment_loss‚7.251564570835659 trainer/global_step‚99 % _timestamp‚1.7883893314085505e+09 /loss/FRt1/detection_loss‚2.9311855452328497 .loss/FRt3/detection_loss‚3.551106483705582 /loss/FBt4/detection_loss‚1.5285830414454014 &loss/total_loss‚105.19374484883173 epoch‚0 0loss/FBt2/assignment_loss‚0.5892138179624453 .loss/FRt4/detection_loss‚5.772245909486498 /loss/FBt3/detection_loss‚12.847684117654959 _runtime‚ 87.164341099 1loss/SRqqt3/assignment_loss‚3.764859844303731 0loss/SRqqt1/detection_loss‚2.953400777445899 0loss/SRqqt4/detection_loss‚7.133405014195225 -loss/FBt2/detection_loss‚9.21218201342751 /loss/FRt2/assignment_loss‚7.333351411317524 2loss/SRqqt4/assignment_loss‚3.7678869529203936 0loss/FBt1/assignment_loss‚0.4590800337884399 1loss/FBt4/assignment_loss‚0.25676869247386386 0loss/SRqqt2/detection_loss‚3.2814862425580174ˆ<‘ÒÑÇ 2loss/SRqqt2/assignment_loss‚3.8212107335678254 1loss/FBt3/assignment_loss‚0.46430827758740634 1loss/SRqqt3/detection_loss‚1.3764205417446074 _step‚7 /loss/FRt3/assignment_loss‚7.406675979655276 1loss/SRqqt1/assignment_loss‚3.347191764248742 .loss/FRt2/detection_loss‚2.966327424504255 .loss/FBt1/detection_loss‚6.029088598939608 +loss/FRt1/assignment_loss‚ 7.14851705983 /loss/FRt4/assignment_loss‚7.251564570835659 ÍL6Q{trainer/global_step‚99 % _timestamp‚1.7883893314085505e+09 /loss/FRt1/detection_loss‚2.9311855452328497 .loss/FRt3/detection_loss‚3.551106483705582 /loss/FBt4/detection_loss‚1.5285830414454014 &loss/total_loss‚105.19374484883173 epoch‚0 0loss/FBt2/assignment_loss‚0.5892138179624453 .loss/FRt4/detection_loss‚5.772245909486498 /loss/FBt3/detection_loss‚12.847684117654959 _runtime‚ 87.164341099 1loss/SRqqt3/assignment_loss‚3.764859844303731 0loss/SRqqt1/detection_loss‚2.953400777445899 0loss/SRqqt4/detection_loss‚7.133405014195225 -loss/FBt2/detection_loss‚9.21218201342751 /loss/FRt2/assignment_loss‚7.333351411317524 2loss/SRqqt4/assignment_loss‚3.7678869529203936 0loss/FBt1/assignment_loss‚0.4590800337884399 1loss/FBt4/assignment_loss‚0.25676869247386386 0loss/SRqqt2/detection_loss‚3.281486242558017¤!u1’Ò‚(:‡ ÖÏâÔã½Æu gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.672' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.1344 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0éAGÓ‚(:ƒ ÖÏâÔ€¬w disk.dm-0.out‚539.9716796875 disk.rbd2.out‚ 4.7158203125 disk.sda.in‚0 disk.sda.out‚ 2.95703125( proc.memory.availableMB‚ 3117.3359375 cpu‚15.314059857051108 disk.drbd1777.out‚0) disk./.usagePercent‚69.95192542037665 disk.rbd1.in‚0 proc.cpu.threads‚17 disk.rbd3.out‚ 0.21142578125% disk.nvme0c0n1.out‚539.9716796875 disk.rbd3.in‚0 disk.dm-0.in‚ 1.83203125) proc.memory.percent‚17.09895133972168 network.sent‚144136 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚ 1.83203125 disk.sda1.in‚0$ memory_percent‚80.97329139709473 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda2.out‚0 disk.sda3.in‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚ 1.83203125# disk.nvme0n1.out‚539.9716796875" proc.memory.rssMB‚ 2801.4921875 network.recv‚97183 disk.dm-1.in‚0 disk.rbd1.out‚0% disk./.usageGB‚1042.3342399597168 disk.sda2.in‚0 disk.sda3.out‚ 2.95703125 disk.dm-1.out‚ 2.95703125 disk.sda1.out‚0HÏö’Ô‚(:‡ åÏâÔ¦ÚÄu gpu.0.gpu‚23.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚122.934' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5868 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Æs¼8ýÕ‚(:ò åÏâÔ±·òw disk.sda3.out‚3.5 disk.rbd0.out‚0 disk.sda2.out‚0% disk./.usageGB‚1042.3342475891113 disk.rbd1.in‚0) proc.memory.availableMB‚ 3117.10546875" proc.memory.rssMB‚ 2801.4921875 disk.rbd3.in‚0 disk.rbd3.out‚ 0.21142578125$ disk.nvme0n1.out‚638.59814453125 disk.rbd0.in‚0 network.recv‚102377 disk.dm-0.in‚ 1.83203125 disk.rbd2.out‚ 5.7275390625 disk.sda1.out‚0 disk.nvme0c0n1.in‚ 1.83203125) proc.memory.percent‚17.09895133972168 proc.cpu.threads‚17 disk.sda1.in‚0 disk.rbd1.out‚0) disk./.usagePercent‚69.95192593239172! disk.dm-0.out‚638.59814453125 disk.dm-1.in‚0 disk.dm-1.out‚3.5 disk.rbd2.in‚0 disk.nvme0n1.in‚ 1.83203125 disk.sda3.in‚0 disk.sda.out‚3.5& disk.nvme0c0n1.out‚638.59814453125 disk.sda2.in‚0 cpu‚16.05443744038405 network.sent‚193394 disk.sda.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ memory_percent‚80.974698066711432½ Ö‚(:’ ôÏâÔ¾÷Žv gpu.0.gpu‚15.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.496' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.699200000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Å*µ¿×‚(:ˆ ôÏâÔ—‚—y! disk.dm-0.out‚736.99658203125 disk.sda.out‚ 3.8359375) disk./.usagePercent‚69.95194385291913 disk.sda1.out‚0 disk.rbd3.out‚ 0.2509765625 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.sda3.in‚0 disk.rbd1.in‚0$ disk.nvme0n1.out‚736.99658203125 disk.sda.in‚0$ memory_percent‚80.97631931304932) proc.memory.availableMB‚ 3116.83984375 disk.sda2.out‚0# disk./.usageGB‚1042.33451461792 disk.sda2.in‚0 disk.nvme0n1.in‚ 1.83203125 proc.cpu.threads‚17& disk.nvme0c0n1.out‚736.99658203125 disk.rbd1.out‚0# proc.memory.rssMB‚ 2801.49609375) proc.memory.percent‚17.09897518157959 network.recv‚107494 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚ 1.83203125 network.sent‚199971 disk.rbd0.in‚0 disk.rbd0.out‚0.0625 disk.sda3.out‚ 3.8359375 disk.dm-1.in‚0 disk.dm-1.out‚ 3.8359375 disk.rbd2.in‚0 disk.sda1.in‚0 disk.dm-0.in‚ 1.83203125 disk.rbd2.out‚ 5.8681640625 cpu‚16.695707997615163ô’0Ø+ _wandbruntime‚128  _runtime‚128áù&nÄÙ¾ ' lr-AdamW/pg1‚0.00042395718957903415 ' lr-AdamW/pg2‚0.00042395718957903415 trainer/global_step‚149 % _timestamp‚1.7883893725428476e+09 _runtime‚ 128.29892318 _step‚8õÈÚ ' lr-AdamW/pg1‚0.00042395718957903415 ' lr-AdamW/pg2‚0.00042395718957903415 trainer/global_step‚149 % _timestamp‚1.7883893725428476e+09 _runtime‚ 128.29892318 _step‚8®é 0Û+ _wandbruntime‚128  _runtime‚128ª1ˆ¤'ÜžO YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 9780632378368 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 120223245312 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2486964 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3134350 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 9560831268864 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2486964 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚9 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2291153 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 124628898304 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2291659 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 9900643067904 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚5621314 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚506 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1512740 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 249079808 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 119541948054 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 9900643067904 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 120027728896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 120027728896 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 9855144099328 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 119349957838 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3155654 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 9900855623680 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚778413 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 128.300174306 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 9780615339008 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚778916 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚503 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 9680373216918 trainer/global_step‚149 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2486966 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1130 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 9560814229504 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2486966 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 9900855623680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824DkU¨'Ý¢O MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2486966 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 9900855623680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 9780632378368 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 120223245312 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2486964 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3134350 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 9560831268864 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2486964 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚9 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2291153 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 124628898304 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2291659 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 9900643067904 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚5621314 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚506 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1512740 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 249079808 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 119541948054 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 9900643067904 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 120027728896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 120027728896 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 9855144099328 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 119349957838 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3155654 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 9900855623680 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚778413 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 128.300174306 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 9780615339008 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚778916 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚503 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 9680373216918 trainer/global_step‚149 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2486966 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1130 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 9560814229504 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 Ðy0Þ+ _wandbruntime‚129  _runtime‚129ZCDyÕß«M R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 9966693064704 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚5658706 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚531 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 120826119680 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3155222 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 9845671415296 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2306451 _step‚10 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 121021649408 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 120143840786 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242775040 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 9845654375936 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267678720 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 9845654375936 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 125708292608 >6DeviceStatsMonitor܉Ü .on_train_batch_end/segment.all.peak‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1130 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3155222 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 9966693064704 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2503484 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚783677 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 9845671415296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 121021649408 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚5680038 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2503484 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 9744557454866 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚5680038 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 9624430653440Ÿâ.µ&à¯M RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 9966480495616 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3176552 trainer/global_step‚149 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 9920694499328 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 9920669595648 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242775040 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 9845654375936 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267678720 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 9845654375936 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 125708292608 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1130 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3155222 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 9966693064704 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2503484 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚783677 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 9845671415296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 121021649408 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚5680038 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2503484 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 9744557454866 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚5680038 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 9624430653440 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 9966693064704 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚5658706 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚531 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 120826119680 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3155222 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 9845671415296 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2306451 _step‚10 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 121021649408 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 120143840786 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 E0í+ _wandbruntime‚170  _runtime‚170Cñ½¶'î°O =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4181302 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4181302 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4202606 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13131418915328 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚12740783458304 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚7494266 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3312964 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚13032584224768 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2014440 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 159071277246 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 159975096320 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3312964 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13131443819008 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚13192754837504 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1131 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 253274112 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1037629 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 160170612736 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚13032584224768 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1037103 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚13192542281728 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚13192542281728 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚13032567185408 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚7515572 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2014443 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚526 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚12899854735550 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚529 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚12900063765126 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚13297497843712 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4202606 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 278177792 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚13192754837504 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 166332202496 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚13 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3052072 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 159263267462 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936Kz{s–ï´O WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1037629 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 160170612736 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚13032584224768 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1037103 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚13192542281728 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚13192542281728 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚13032567185408 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚7515572 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2014443 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚526 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚12899854735550 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚529 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚12900063765126 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚13297497843712 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4202606 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 278177792 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚13192754837504 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 166332202496 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚13 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3052072 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 159263267462 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4181302 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4181302 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4202606 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13131418915328 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚12740783458304 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚7494266 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3312964 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚13032584224768 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2014440 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 159071277246 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 159975096320 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3312964 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13131443819008 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚13192754837504 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1131 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 253274112 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ¢ð‘U0ð+ _wandbruntime‚171  _runtime‚171ÈÉÁ&ñ»M R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚13258379709440 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚13196944411648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚13363850635264 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚13258592278528 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚13097606222336 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 160773487104 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3329484 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 160773487104 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13097623261696 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 166906223616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3066883 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚199 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚7531658 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚7531658 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚14 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3329486 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4223504 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 167157387264 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1131 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚13364126702592 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚7552990 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13097623261696 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7883894154836655e+09 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3329486 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚13196944411648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚13363850635264 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚13258592278528 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚13097606222336 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 160773487104 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3329484 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 160773487104 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13097623261696 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 166906223616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3066883 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚199 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚7531658 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚7531658 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚14 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3329486 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4223504 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 167157387264 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1131 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚13364126702592 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚7552990 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13097623261696 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7883894154836655e+09 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3329486 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 íî0ó+ _wandbruntime‚171  _runtime‚171^’“dPôÊ /loss/FRt3/assignment_loss‚7.406684005961699 0loss/SRqqt1/detection_loss‚2.924093703834378 1loss/SRqqt4/detection_loss‚3.6316318564777132 /loss/FBt4/detection_loss‚0.9963069925010964 /loss/FRt2/assignment_loss‚5.243558426292575 .loss/FRt4/assignment_loss‚7.10147860315111 2loss/SRqqt3/assignment_loss‚3.1370834485515133 1loss/FBt1/assignment_loss‚0.49141135323038837 1loss/FBt2/assignment_loss‚0.29223407649695016 .loss/FBt3/assignment_loss‚0.52318740638449 /loss/FBt4/assignment_loss‚0.640813580370074 1loss/SRqqt3/detection_loss‚1.8957074872550557 2loss/SRqqt1/assignment_loss‚3.1002469865643247 2loss/SRqqt2/assignment_loss‚1.8157758247876956 .loss/FRt3/detection_loss‚5.770582234742594 .loss/FBt2/detection_loss‚3.627455501920647 trainer/global_step‚199 %loss/total_loss‚80.06936402253301 epoch‚0 _step‚15 /loss/FRt2/detection_loss‚2.2218068348509923 0loss/SRqqt2/detection_loss‚2.571322962456991 _runtime‚ 171.241422763 /loss/FRt1/assignment_loss‚6.707319553862226 1loss/SRqqt4/assignment_loss‚2.865279301057888 /loss/FRt4/detection_loss‚4.3673819700876875 .loss/FBt3/detection_loss‚6.317964470812252 % _timestamp‚1.7883894154856021e+09 /loss/FRt1/detection_loss‚3.1876935797763313 /loss/FBt1/detection_loss‚3.2323438611063287Ï×BTõÎ /loss/FRt3/assignment_loss‚7.406684005961699 0loss/SRqqt1/detection_loss‚2.924093703834378 1loss/SRqqt4/detection_loss‚3.6316318564777132 /loss/FBt4/detection_loss‚0.9963069925010964 /loss/FRt2/assignment_loss‚5.243558426292575 .loss/FRt4/assignment_loss‚7.10147860315111 2loss/SRqqt3/assignment_loss‚3.1370834485515133 1loss/FBt1/assignment_loss‚0.49141135323038837 1loss/FBt2/assignment_loss‚0.29223407649695016 .loss/FBt3/assignment_loss‚0.52318740638449 /loss/FBt4/assignment_loss‚0.640813580370074 1loss/SRqqt3/detection_loss‚1.8957074872550557 2loss/SRqqt1/assignment_loss‚3.1002469865643247 2loss/SRqqt2/assignment_loss‚1.8157758247876956 .loss/FRt3/detection_loss‚5.770582234742594 .loss/FBt2/detection_loss‚3.627455501920647 trainer/global_step‚199 %loss/total_loss‚80.06936402253301 epoch‚0 _step‚15 /loss/FRt2/detection_loss‚2.2218068348509923 0loss/SRqqt2/detection_loss‚2.571322962456991 _runtime‚ 171.241422763 /loss/FRt1/assignment_loss‚6.707319553862226 1loss/SRqqt4/assignment_loss‚2.865279301057888 /loss/FRt4/detection_loss‚4.3673819700876875 .loss/FBt3/detection_loss‚6.317964470812252 % _timestamp‚1.7883894154856021e+09 /loss/FRt1/detection_loss‚3.1876935797763313 /loss/FBt1/detection_loss‚3.2323438611063287và¼Ïœö‚(:‘ °ÐâÔý¢w gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚153.685' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.737000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ø‚ç!÷‚(:ø °ÐâÔ’ÕÑy( proc.memory.availableMB‚ 3116.5703125 disk.nvme0n1.in‚ 33.140625 disk.sda2.out‚0 disk.rbd3.out‚ 0.376953125 cpu‚18.585721181061 disk.sda3.out‚ 5.96484375 disk.rbd1.in‚0 disk.sda.in‚0$ disk.nvme0c0n1.out‚ 935.603515625 disk.rbd3.in‚0 proc.cpu.threads‚17 disk.rbd2.out‚ 6.87255859375 disk.sda.out‚ 5.96484375 disk.dm-0.in‚ 33.140625( proc.memory.percent‚17.0989990234375 disk.dm-1.in‚0$ memory_percent‚80.97796440124512 disk.rbd0.out‚0.078125 network.sent‚268973( disk./.usagePercent‚69.9520229592472 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.dm-0.out‚ 935.603515625 proc.memory.rssMB‚2801.5 network.recv‚127680$ disk./.usageGB‚1042.335693359375 disk.sda3.in‚0 disk.dm-1.out‚ 5.96484375 disk.rbd1.out‚0 disk.rbd0.in‚0 disk.rbd2.in‚0" disk.nvme0n1.out‚ 935.603515625 disk.sda1.in‚0 disk.sda1.out‚0 disk.nvme0c0n1.in‚ 33.140625ÿ+…´mø‚(:† ¿ÐâÔòÌÍu gpu.0.gpu‚7.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚14.096755981445314UÔŽè$* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚136.337' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.2674 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)¦{¹ ù‚(:‚ ¿ÐâÔå°Žy disk.rbd2.in‚0$ disk.nvme0n1.out‚1034.0615234375 disk.sda1.in‚0 disk.nvme0c0n1.in‚34.875# proc.memory.rssMB‚ 2801.50390625 cpu‚18.940565605662968 disk.sda2.out‚0 disk.drbd1777.out‚0 network.sent‚319424 disk.sda2.in‚0& disk.nvme0c0n1.out‚1034.0615234375) proc.memory.percent‚17.09902286529541) disk./.usagePercent‚69.95202423928487 disk.rbd3.out‚ 0.376953125 disk.sda3.out‚ 6.33203125 disk.rbd1.in‚0 disk.rbd2.out‚ 6.87646484375 network.recv‚133873 disk.rbd3.in‚0! disk.dm-0.out‚1034.0615234375 disk.dm-1.out‚ 6.33203125 disk.rbd1.out‚0 disk.nvme0n1.in‚34.875$ memory_percent‚80.97987174987793( proc.memory.availableMB‚ 3116.2578125 disk.sda3.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0% disk./.usageGB‚1042.3357124328613 disk.sda.in‚0 disk.rbd0.out‚0.078125 disk.dm-0.in‚34.875 disk.sda.out‚ 6.33203125 disk.sda1.out‚0 proc.cpu.threads‚17 disk.dm-1.in‚0i¯’ú‚(:‡ ÎÐâÔ¥Ùu gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.941' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.3882 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0–Aû‚(:÷ ÎÐâÔ¥ðâw disk.dm-1.out‚7.09375 disk.rbd2.out‚ 7.017578125 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda3.in‚0 network.recv‚138070 disk.sda.in‚0 disk.sda1.in‚0 disk.rbd3.in‚0 proc.cpu.threads‚17) disk./.usagePercent‚69.95203396757118 disk.dm-1.in‚0 disk.sda.out‚7.09375 network.sent‚325156 disk.rbd2.in‚0& disk.nvme0c0n1.out‚1034.4130859375 disk.rbd3.out‚ 0.439453125 disk.sda3.out‚7.09375% disk./.usageGB‚1042.3358573913574 disk.nvme0n1.in‚34.875 disk.nvme0c0n1.in‚34.875" proc.memory.rssMB‚ 2801.5078125 disk.rbd1.out‚0 disk.drbd1777.out‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 3116.16796875 cpu‚19.259267953095204 disk.rbd1.in‚0 disk.rbd0.out‚0.078125 disk.dm-0.in‚34.875! disk.dm-0.out‚1034.4130859375) proc.memory.percent‚17.09904670715332$ disk.nvme0n1.out‚1034.4130859375 disk.rbd0.in‚0 disk.drbd1777.in‚0$ memory_percent‚80.98042011260986']PG0ü+ _wandbruntime‚212  _runtime‚212ðVJÉÄý¾ & lr-AdamW/pg1‚0.0007084922161421442 & lr-AdamW/pg2‚0.0007084922161421442 trainer/global_step‚249 % _timestamp‚1.7883894568115616e+09 _runtime‚ 212.567519681 _step‚16Ó³Èþ _step‚16 & lr-AdamW/pg1‚0.0007084922161421442 & lr-AdamW/pg2‚0.0007084922161421442 trainer/global_step‚249 % _timestamp‚1.7883894568115616e+09 _runtime‚ 212.567519681ˆ@Ûü0ÿ+ _wandbruntime‚212  _runtime‚212m–4Ù·'€±O KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 200117980160 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1138 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚16615473948160 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _step‚17 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚5249558 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 198984586870 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚16284536071168 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4138964 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚16484654051328 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚9388524 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883894568126094e+09 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16284519031808 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 253274112 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 200117980160 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2516143 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1120 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16484441495552 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚15920769726464 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 199922463744 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16484441495552 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4138964 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16407718635008 _runtime‚ 212.571016678 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚16284519031808 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 207755313152 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5249558 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 198792596654 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4138966 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1295896 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15920752687104 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16615195770368 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚16407693731328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚16484654051328 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3812036 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5228254 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3812578 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1296435 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚16119545283758 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 199922463744 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 207502039040 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9367218 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507[ÕÙ»'µO MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 207502039040 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9367218 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 200117980160 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1138 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚16615473948160 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _step‚17 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚5249558 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 198984586870 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚16284536071168 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4138964 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚16484654051328 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚9388524 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883894568126094e+09 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16284519031808 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 253274112 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 200117980160 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2516143 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1120 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16484441495552 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚15920769726464 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 199922463744 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16484441495552 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4138964 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16407718635008 _runtime‚ 212.571016678 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚16284519031808 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 207755313152 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5249558 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 198792596654 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4138966 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1295896 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15920752687104 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16615195770368 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚16407693731328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚16484654051328 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3812036 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 266401280 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5228254 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3812578 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1296435 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚16119545283758 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 199922463744¤CÉ.0‚+ _wandbruntime‚213  _runtime‚213¡¿ŸY#ƒ»M JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5270456 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1301123 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1120 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1301648 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚16184147581066 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚16183938551282 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4155486 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242775040 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9425942 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 208334231040 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5249126 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚15984352071680 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚18 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 199586479602 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚9404610 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚5249126 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚16473219227648 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚16550491492352 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 % _timestamp‚1.7883894575766845e+09 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267678720 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 200720854528 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16349575108096 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4155484 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2526177 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2526174 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 208577006080 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16473244131328 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16550491492352 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3827297 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 200720854528 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 _runtime‚ 213.333203769 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚528 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 199778470026 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4155486 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16550278923264 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4155484 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16349558068736 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 200916384256 MCDeviceStatsMonitor.on_train_batch_end/allocation.smal͸3Ühl_pool.current‚21330 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9425942 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚5270456 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3827825 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1138 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚525 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16681821137408 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16349558068736 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 200916384256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9425942 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚5270456 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3827825 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1138 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚525 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16681821137408 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5270456 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1301123 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1120 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1301648 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚16184147581066 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚16183938551282 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4155486 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242775040 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9425942 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 208334231040 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5249126 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚15984352071680 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚18 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 199586479602 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚9404610 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚5249126 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚16473219227648 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚16550491492352 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 % _timestamp‚1.7883894575766845e+09 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267678720 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 200720854528 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16349575108096 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4155484 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2526177 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2526174 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 208577006080 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16473244131328 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16550491492352 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3827297 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 200720854528 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 _runtime‚ 213.333203769 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚528 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 199778470026 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4155486 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16550278923264 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4155484 FDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11240170 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4573156 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19776340709376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11261476 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4964966 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚19100738955264 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚19339235831966 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 238513916062 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚19776553265152 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 239869831168 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19683993451008 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 239869831168 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3017843 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 240065347584 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19776340709376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11261476 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4964966 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚19100738955264 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚19339235831966 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 238513916062 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚19776553265152 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 239869831168 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19683993451008 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 239869831168 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3017843 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 240065347584 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11240170 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4573156 Ið’‚(:’ ûÐâÔÅ»¨u gpu.0.gpu‚60.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.793' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.358600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0€' “‚(:‘ ûÐâÔŠÖÑw% disk.nvme0c0n1.out‚1233.169921875 network.sent‚388104 disk.sda.out‚ 8.58984375! disk.nvme0c0n1.in‚ 34.91796875 disk.rbd0.in‚0 disk.drbd1777.in‚0 proc.cpu.threads‚17 network.recv‚153846 disk.drbd1777.out‚0 disk.dm-0.in‚ 34.91796875$ memory_percent‚80.98125457763672 disk.rbd0.out‚0.078125 disk.rbd3.in‚0 disk.sda3.in‚0# disk.nvme0n1.out‚1233.169921875& proc.memory.availableMB‚ 3116.03125( disk./.usagePercent‚69.9520342235787 disk.sda2.out‚0 disk.rbd2.in‚0 cpu‚20.060892172800703 disk.sda2.in‚0 disk.rbd3.out‚ 0.705078125 disk.dm-0.out‚1233.169921875 disk.sda3.out‚ 8.58984375 disk.rbd2.out‚ 7.083984375# proc.memory.rssMB‚ 2801.51171875% disk./.usageGB‚1042.3358612060547 disk.sda.in‚0 disk.dm-1.out‚ 8.58984375 disk.rbd1.in‚0 disk.rbd1.out‚ 0.1484375 disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.nvme0n1.in‚ 34.91796875) proc.memory.percent‚17.09907054901123ÕWµý0”+ _wandbruntime‚255  _runtime‚255èJEÅ&•¿M XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 266401280 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚6296078 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19601509915136 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19842390706176 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 240668221952 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 249066496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 239499789434 trainer/global_step‚299 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚19842390706176 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11298894 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6317408 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1121 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4981486 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 240863751680 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11298894 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19601509915136 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1139 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _step‚22 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚509 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4981484 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚19749518947328 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚ 255.412334916 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚19601526954496 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4981486 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3027874 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4981484 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4587715 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚11277562 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4588227 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11277562 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6317408 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273970176 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚19999214679040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚19164321300480 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 240668221952 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚19999488649216 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚19999488649216 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19601509915136 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19842390706176 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 240668221952 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 249066496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 239499789434 trainer/global_step‚299 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚19842390706176 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11298894 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6317408 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1121 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4981486 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 240863751680 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11298894 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19601509915136 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1139 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _step‚22 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚509 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4981484 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚19749518947328 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚ 255.412334916 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚19601526954496 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4981486 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3027874 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4981484 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4587715 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚11277562 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4588227 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11277562 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6317408 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273970176 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚19999214679040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚19164321300480 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 240668221952zJ>F0—+ _wandbruntime‚255  _runtime‚255¡ ¿QO˜É 0loss/FBt4/assignment_loss‚0.5628860069868656 /loss/FBt1/detection_loss‚2.7951790265376433 2loss/SRqqt2/assignment_loss‚1.8949737463678633 2loss/SRqqt3/assignment_loss‚2.9248650761751027 .loss/FBt3/detection_loss‚4.704056993774746 %loss/total_loss‚72.70168927149072 epoch‚0 /loss/FRt3/assignment_loss‚7.153119336475026 /loss/FRt3/detection_loss‚3.5081004148179833 0loss/SRqqt2/detection_loss‚1.726142454166691 0loss/SRqqt3/detection_loss‚3.066966854608976 trainer/global_step‚299 2loss/SRqqt1/assignment_loss‚3.1387601904912827 .loss/FBt4/detection_loss‚5.904768742047823 /loss/FBt3/assignment_loss‚0.530306649072476 /loss/FRt1/assignment_loss‚6.370836544036865 /loss/FRt4/assignment_loss‚6.440825783476537 0loss/SRqqt4/assignment_loss‚2.78225083671399 .loss/FRt1/detection_loss‚2.412024057479132 /loss/FRt2/detection_loss‚1.9018782011203228 _runtime‚ 255.413441581 _step‚23 0loss/FBt1/assignment_loss‚0.6283954388791437 0loss/FBt2/assignment_loss‚0.2856140388392381 /loss/FRt4/detection_loss‚2.7969589136084734 1loss/SRqqt1/detection_loss‚2.1107760362942285 0loss/SRqqt4/detection_loss‚2.983610361369688 /loss/FBt2/detection_loss‚1.4599158444752296 /loss/FRt2/assignment_loss‚4.618477723675389 % _timestamp‚1.7883894996576498e+09f+ÈS™Í /loss/FRt3/detection_loss‚3.5081004148179833 0loss/SRqqt2/detection_loss‚1.726142454166691 0loss/SRqqt3/detection_loss‚3.066966854608976 trainer/global_step‚299 2loss/SRqqt1/assignment_loss‚3.1387601904912827 .loss/FBt4/detection_loss‚5.904768742047823 /loss/FBt3/assignment_loss‚0.530306649072476 /loss/FRt1/assignment_loss‚6.370836544036865 /loss/FRt4/assignment_loss‚6.440825783476537 0loss/SRqqt4/assignment_loss‚2.78225083671399 .loss/FRt1/detection_loss‚2.412024057479132 /loss/FRt2/detection_loss‚1.9018782011203228 _runtime‚ 255.413441581 _step‚23 0loss/FBt1/assignment_loss‚0.6283954388791437 0loss/FBt2/assignment_loss‚0.2856140388392381 /loss/FRt4/detection_loss‚2.7969589136084734 1loss/SRqqt1/detection_loss‚2.1107760362942285 0loss/SRqqt4/detection_loss‚2.983610361369688 /loss/FBt2/detection_loss‚1.4599158444752296 /loss/FRt2/assignment_loss‚4.618477723675389 % _timestamp‚1.7883894996576498e+09 0loss/FBt4/assignment_loss‚0.5628860069868656 /loss/FBt1/detection_loss‚2.7951790265376433 2loss/SRqqt2/assignment_loss‚1.8949737463678633 2loss/SRqqt3/assignment_loss‚2.9248650761751027 .loss/FBt3/detection_loss‚4.704056993774746 %loss/total_loss‚72.70168927149072 epoch‚0 /loss/FRt3/assignment_loss‚7.153119336475026ÝyN’š‚(:‡ ŠÑâÔœ†¼u gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.264' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.2528 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0>@]›‚(:Œ ŠÑâÔ…ø›x) proc.memory.percent‚17.09909439086914 proc.cpu.threads‚17 network.recv‚158849 disk.sda2.in‚0 disk.rbd3.out‚ 0.798828125 disk.sda3.in‚0 disk.dm-1.out‚ 9.3046875 disk.nvme0n1.in‚ 34.91796875 cpu‚20.285428913532325 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.dm-0.out‚ 1331.69140625 disk.sda1.in‚0$ memory_percent‚80.98292350769043 disk.rbd1.out‚ 0.1484375 disk.rbd2.in‚0 disk.rbd2.out‚ 7.146484375) disk./.usagePercent‚69.95203601563145 disk.rbd0.in‚0 disk.sda2.out‚0( proc.memory.availableMB‚ 3115.7578125! disk.nvme0c0n1.in‚ 34.91796875$ disk.nvme0c0n1.out‚ 1331.69140625 disk.rbd1.in‚0" disk.nvme0n1.out‚ 1331.69140625 disk.sda1.out‚0! proc.memory.rssMB‚ 2801.515625 network.sent‚425966 disk.dm-1.in‚0 disk.sda.out‚ 9.3046875% disk./.usageGB‚1042.3358879089355 disk.drbd1777.in‚0 disk.dm-0.in‚ 34.91796875 disk.sda3.out‚ 9.3046875 disk.sda.in‚0Þ €œ‚(:’ ™ÑâÔù¸žu gpu.0.gpu‚16.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.432' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.486399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0´×i(‚(: ™ÑâÔÙ™Øx disk.rbd3.out‚ 0.798828125" disk.dm-0.out‚1336.44873046875 disk.sda1.out‚0 disk.rbd3.in‚0 disk.sda3.in‚0 disk.dm-1.in‚0( proc.memory.availableMB‚ 3115.7421875 disk.sda.out‚ 9.94140625 disk.rbd0.in‚0 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚1336.44873046875$ memory_percent‚80.98301887512207 network.sent‚464780% disk./.usageGB‚1042.3345031738281! disk.nvme0c0n1.in‚ 34.95703125 disk.dm-1.out‚ 9.94140625# proc.memory.rssMB‚ 2801.51953125 cpu‚20.492678182274943 network.recv‚163426 disk.sda1.in‚0 disk.dm-0.in‚ 34.95703125 disk.sda3.out‚ 9.94140625 disk.rbd1.out‚ 0.1806640625% disk.nvme0n1.out‚1336.44873046875 disk.sda.in‚0 disk.sda2.in‚0 proc.cpu.threads‚17 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.rbd1.in‚0) proc.memory.percent‚17.09911823272705) disk./.usagePercent‚69.95194308489651 disk.rbd2.in‚0 disk.rbd2.out‚ 7.146484375 disk.nvme0n1.in‚ 34.95703125 disk.drbd1777.in‚0A³±—0ž+ _wandbruntime‚296  _runtime‚2967­èÄŸ¾ trainer/global_step‚349 % _timestamp‚1.7883895408998451e+09 _runtime‚ 296.655872267 _step‚24 & lr-AdamW/pg1‚0.0009930272427052544 & lr-AdamW/pg2‚0.0009930272427052544Ô`ÚÈ Â & lr-AdamW/pg1‚0.0009930272427052544 & lr-AdamW/pg2‚0.0009930272427052544 trainer/global_step‚349 % _timestamp‚1.7883895408998451e+09 _runtime‚ 296.655872267 _step‚24®)ëÌ0¡+ _wandbruntime‚296  _runtime‚296é8?°¿¢´O QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚520 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚73HÉY†û%22158 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 280012715008 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3519540 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚22280708184064 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5790966 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 $ _timestamp‚1.788389540900903e+09 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚13134428 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7343462 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7322158 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚22788422724608 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _step‚25 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚23250836474368 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22960243363328 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 278235235470 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚22559135409750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5790966 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚23250552005120 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚13113122 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚23068239923200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _runtime‚ 296.657120188 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7343462 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3519543 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5790964 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚22558926380174 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 278427225686 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚349 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1812945 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚22280691144704 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5332485 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1122 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 280012715008 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1140 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚23068239923200 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5790964 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚22960268267008 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1813465 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 290568207360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚22788439763968 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 284469248 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7322158 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 280012715008 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3519540 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚22280708184064 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5790966 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 $ _timestamp‚1.788389540900903e+09 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚13134428 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7343462 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7322158 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚22788422724608 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 355378688 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _step‚25 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚23250836474368 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22960243363328 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 278235235470 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚22559135409750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5790966 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚23250552005120 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚13113122 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚23068239923200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _runtime‚ 296.657120188 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7343462 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3519543 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5790964 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚22558926380174 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 278427225686 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚349 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1812945 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚22280691144704 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5332485 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1122 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 280012715008 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1140 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚23068239923200 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5790964 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚22960268267008 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1813465 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 290568207360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚22788439763968 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 284469248 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523йâ{0¤+ _wandbruntime‚297  _runtime‚297§PºÅ&¥¿M NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5348257 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 257455104 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚22853461761536 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 280811119104 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚23316898341376 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 280615589376 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚23134289920000 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1122 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _runtime‚ 297.403029683 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚5807486 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚23025793763328 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 279221108842 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5807486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚23134289920000 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚22853478800896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7364360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3529574 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚13150514 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 279029118418 trainer/global_step‚349 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 280615589376 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1818183 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1818680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚23134077350912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚22853461761536 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚13150514 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚23025768859648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚22344307568640 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 280811119104 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1140 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚22623528677482 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚26 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13171846 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3529577 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 282358784 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5347757 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚497 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7883895416464088e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5807484 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚13171846 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚23025793763328 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 279221108842 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5807486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.al ®<šl.allocated‚23134289920000 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚22853478800896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7364360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3529574 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚13150514 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 279029118418 trainer/global_step‚349 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 280615589376 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1818183 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1818680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚23134077350912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚22853461761536 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚13150514 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 355378688 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚23025768859648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚22344307568640 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 280811119104 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1140 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚22623528677482 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚26 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13171846 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3529577 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 282358784 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5347757 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚497 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7883895416464088e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5807484 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚13171846 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 EDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 318148545094 % _timestamp‚1.7883895826134312e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 319764566016 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4021243 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269789184 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 317956554878 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6616966 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚26568485385216 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8369110 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 319960082432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚26360139137024 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1122 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2073347 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚26236518179328 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 331942302208 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚26360351692800 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6616966 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14986074 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1140 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚26040374571008 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 331697416704 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6094590 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚25460677412864 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _runtime‚ 338.369688819 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15007380 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6616964 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8369110 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 319764566016 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244885504 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 trainer/global_step‚399 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6094051 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚26360139137024 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8390414 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6616964 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚26040374571008ž#JŒJµ¹O D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚26236543083008 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚26040391610368 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2072811 _step‚29 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚26040391610368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 318148545094 % _timestamp‚1.7883895826134312e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 319764566016 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4021243 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269789184 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 317956554878 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6616966 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚26568485385216 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8369110 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 319960082432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚26360139137024 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1122 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2073347 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚26236518179328 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 331942302208 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚26360351692800 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6616966 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14986074 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1140 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚26040374571008 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 331697416704 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6094590 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚25460677412864 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _runtime‚ 338.369688819 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15007380 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6616964 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8369110 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 319764566016 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244885504 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 trainer/global_step‚399 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6094051 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚26360139137024 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8390414 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6616964 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚26040374571008 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚26568215596032 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚25778616928382 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚25460660373504 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824ƒœÈ0¶+ _wandbruntime‚339  _runtime‚339Œ…ÙªÅ&·¿M VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚26105413607936 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚26105430647296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 % _timestamp‚1.7883895833559406e+09 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚26634576249344 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6633484 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2078066 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚25524276797440 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚26426189133824 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚25524259758080 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8389982 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8389982 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚26634837636608 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 332769057280 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6109340 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚25843219225690 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚399 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚26105430647296 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6633486 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2078593 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚26302043675648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15044798 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15044798 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 261387264 _runtime‚ 339.112471664 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 332532573696 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚30 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4031274 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 318942428250 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 236483584 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 320562956800 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚15023466 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6633484 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8411312 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 320758486528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚15023466 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 320562956800 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389GÈ,ÐÉ&¸ÃM I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8389982 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8389982 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚26634837636608 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 332769057280 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6109340 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚25843219225690 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚399 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚26105430647296 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6633486 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2078593 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚26302043675648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15044798 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15044798 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 261387264 _runtime‚ 339.112471664 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 332532573696 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚30 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4031274 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 318942428250 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 236483584 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 320562956800 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚15023466 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6633484 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8411312 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 320758486528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚15023466 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 320562956800 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚26105413607936 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚26105430647296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 % _timestamp‚1.7883895833559406e+09 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚26634576249344 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6633484 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2078066 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚25524276797440 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚26426189133824 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚25524259758080 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287Sý&ð0¹+ _wandbruntime‚339  _runtime‚339r¡ÚMºÇ .loss/FBt3/detection_loss‚2.791104857960055 % _timestamp‚1.7883895833578398e+09 /loss/FRt3/assignment_loss‚6.064816035722432 /loss/FRt4/assignment_loss‚6.648731488447923 1loss/SRqqt3/assignment_loss‚2.798271135280007 0loss/FBt4/assignment_loss‚0.7083527990616858 /loss/FBt2/detection_loss‚0.9207455246470913 /loss/FBt4/detection_loss‚3.1615076269954443 0loss/SRqqt4/assignment_loss‚2.74286400154233 -loss/FRt3/detection_loss‚3.47625186882521 /loss/FRt4/detection_loss‚2.9434874607966495 .loss/FBt1/detection_loss‚4.408902532914105 2loss/SRqqt1/assignment_loss‚3.1077366269240945 .loss/FRt2/detection_loss‚1.607440077996516 0loss/SRqqt3/detection_loss‚2.844465612842326 1loss/SRqqt2/detection_loss‚1.3860842867236998 trainer/global_step‚399 0loss/SRqqt2/assignment_loss‚1.83062537373561 0loss/FBt1/assignment_loss‚0.5373103261432227 1loss/FBt3/assignment_loss‚0.36421566924954896 /loss/FRt1/detection_loss‚2.7661825587620608 %loss/total_loss‚66.14574566668801 epoch‚0 _runtime‚ 339.113671364 _step‚31 /loss/FRt1/assignment_loss‚5.998522346084182 0loss/FRt2/assignment_loss‚3.9485934566665484 0loss/FBt2/assignment_loss‚0.3403096523680049 1loss/SRqqt1/detection_loss‚2.0167397222781585 /loss/SRqqt4/detection_loss‚2.73248462472111„I»Q»Ë .loss/FBt3/detection_loss‚2.791104857960055 % _timestamp‚1.7883895833578398e+09 /loss/FRt3/assignment_loss‚6.064816035722432 /loss/FRt4/assignment_loss‚6.648731488447923 1loss/SRqqt3/assignment_loss‚2.798271135280007 0loss/FBt4/assignment_loss‚0.7083527990616858 /loss/FBt2/detection_loss‚0.9207455246470913 /loss/FBt4/detection_loss‚3.1615076269954443 0loss/SRqqt4/assignment_loss‚2.74286400154233 -loss/FRt3/detection_loss‚3.47625186882521 /loss/FRt4/detection_loss‚2.9434874607966495 .loss/FBt1/detection_loss‚4.408902532914105 2loss/SRqqt1/assignment_loss‚3.1077366269240945 .loss/FRt2/detection_loss‚1.607440077996516 0loss/SRqqt3/detection_loss‚2.844465612842326 1loss/SRqqt2/detection_loss‚1.3860842867236998 trainer/global_step‚399 0loss/SRqqt2/assignment_loss‚1.83062537373561 0loss/FBt1/assignment_loss‚0.5373103261432227 1loss/FBt3/assignment_loss‚0.36421566924954896 /loss/FRt1/detection_loss‚2.7661825587620608 %loss/total_loss‚66.14574566668801 epoch‚0 _runtime‚ 339.113671364 _step‚31 /loss/FRt1/assignment_loss‚5.998522346084182 0loss/FRt2/assignment_loss‚3.9485934566665484 0loss/FBt2/assignment_loss‚0.3403096523680049 1loss/SRqqt1/detection_loss‚2.0167397222781585 /loss/SRqqt4/detection_loss‚2.732484624721113ŸËἂ(:… ÕÑâÔáõÊu gpu.0.gpu‚59.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.405' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.881 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý'|¡$½‚(:™ ÕÑâÔᲃy) disk./.usagePercent‚69.95208696113076$ disk.nvme0n1.out‚1636.7939453125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.sda3.out‚ 12.76171875" proc.memory.rssMB‚ 2801.5234375 proc.cpu.threads‚17% disk./.usageGB‚1042.3366470336914 disk.dm-1.in‚0 disk.drbd1777.out‚0 disk.sda.in‚0! disk.dm-0.out‚1636.7939453125 disk.rbd1.in‚0 disk.rbd2.in‚0 cpu‚21.18234361740074 network.sent‚532432& disk.nvme0c0n1.out‚1636.7939453125 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚80.97913265228271) proc.memory.percent‚17.09914207458496 disk.rbd1.out‚ 0.306640625 disk.drbd1777.in‚0 network.recv‚181987 disk.nvme0n1.in‚ 38.9921875) proc.memory.availableMB‚ 3116.37890625 disk.nvme0c0n1.in‚ 38.9921875 disk.rbd3.out‚ 0.9248046875 disk.dm-0.in‚ 38.9921875 disk.sda3.in‚0 disk.dm-1.out‚ 12.76171875 disk.rbd2.out‚ 9.451171875 disk.sda.out‚ 12.76171875 disk.sda1.out‚0 disk.rbd3.in‚0g‡ß‘¾‚(:† äÑâÔ”ó½u gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚95.773' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.1546 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)#P‰!¿‚(:– äÑâÔ¯ˆ‡x network.recv‚188124 disk.rbd1.in‚0 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2801.5234375 disk.sda.in‚0 disk.nvme0c0n1.in‚ 39.7109375 disk.rbd3.in‚0( proc.memory.availableMB‚ 3116.4609375 proc.cpu.threads‚17 disk.dm-1.in‚0 disk.rbd0.out‚0.078125# memory_percent‚80.9786319732666% disk./.usageGB‚1042.3368911743164 disk.nvme0n1.in‚ 39.7109375# disk.nvme0n1.out‚1637.232421875 disk.sda.out‚ 13.08984375 disk.drbd1777.in‚0 disk.rbd3.out‚ 0.9248046875 disk.sda3.in‚0 network.sent‚582596 disk.dm-0.out‚1637.232421875 disk.rbd2.out‚ 10.634765625 disk.sda1.out‚0) proc.memory.percent‚17.09914207458496 disk.dm-1.out‚ 13.08984375 disk.rbd2.in‚0 disk.sda3.out‚ 13.08984375 cpu‚21.326745148028692) disk./.usagePercent‚69.95210334561295 disk.rbd1.out‚ 0.306640625 disk.rbd0.in‚0% disk.nvme0c0n1.out‚1637.232421875 disk.sda2.out‚0 disk.dm-0.in‚ 39.7109375ñfK8œÀ‚(:‘ óÑâÔœéu gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚89.693' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.938599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:¡=·"Á‚(:— óÑâÔóÏÕw network.recv‚193251 disk.dm-0.out‚1735.705078125# disk.nvme0n1.out‚1735.705078125 disk.sda.out‚ 13.89453125 disk.drbd1777.out‚0 disk.sda2.out‚0) disk./.usagePercent‚69.95210846576364 disk.rbd3.out‚ 0.9248046875 proc.cpu.threads‚17 disk.dm-0.in‚ 39.7109375 disk.dm-1.out‚ 13.89453125 disk.rbd1.in‚0 disk.rbd0.out‚0.078125) proc.memory.availableMB‚ 3116.47265625 disk.sda3.in‚0 disk.dm-1.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 13.89453125 disk.rbd1.out‚ 0.306640625 disk.rbd2.out‚ 11.646484375) proc.memory.percent‚17.09914207458496 network.sent‚589502% disk./.usageGB‚1042.3369674682617 disk.rbd3.in‚0 disk.sda1.in‚0 disk.nvme0c0n1.in‚ 39.7109375% disk.nvme0c0n1.out‚1735.705078125$ memory_percent‚80.97856044769287" proc.memory.rssMB‚ 2801.5234375 disk.nvme0n1.in‚ 39.7109375 disk.sda.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.rbd2.in‚0 disk.rbd0.in‚0 cpu‚21.46257847102919`Æþ{0Â+ _wandbruntime‚379  _runtime‚3797“gÄþ trainer/global_step‚449 % _timestamp‚1.7883896240723085e+09 _runtime‚ 379.828240674 _step‚32 & lr-AdamW/pg1‚0.0012775622692683644 & lr-AdamW/pg2‚0.0012775622692683644¹½GœÈÄ _runtime‚ 379.828240674 _step‚32 & lr-AdamW/pg1‚0.0012775622692683644 & lr-AdamW/pg2‚0.0012775622692683644 trainer/global_step‚449 % _timestamp‚1.7883896240723085e+09 ðà¾0Å+ _wandbruntime‚379  _runtime‚379Ѹ9Zq ƵO JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 359711933440 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚28640646641664 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7442964 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚16880332 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚494 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9437366 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6854945 % _timestamp‚1.7883896240732636e+09 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29512817899008 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚29885914816512 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 373370900992 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7442964 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚29652038350848 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273983488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 357677874286 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚29292343456768 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚28998516506166 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 359907449856 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 359711933440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚497 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 373121821184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9416062 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚29652038350848 _step‚33 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7442966 _runtime‚ 379.832987839 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚29652250906624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4522943 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 trainer/global_step‚449 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚29292343456768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 249079808 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6855442 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚29512792995328 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚29292326417408 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚28998307476590 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚29292326417408 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚28640629602304 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9437366 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2332499ŽiŽ×¿'ǹO TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚29292326417408 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 267950592 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚28640629602304 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9437366 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2332499 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 359711933440 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚28640646641664 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7442964 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚16880332 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚494 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9437366 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6854945 % _timestamp‚1.7883896240732636e+09 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29512817899008 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚29885914816512 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 373370900992 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7442964 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚29652038350848 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273983488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 357677874286 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚29292343456768 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚28998516506166 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 359907449856 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 359711933440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚497 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 373121821184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9416062 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚29652038350848 _step‚33 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7442966 _runtime‚ 379.832987839 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚29652250906624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4522943 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 trainer/global_step‚449 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚29292343456768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 249079808 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6855442 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚29512792995328 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚29292326417408 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚28998307476590 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936!—ξ0È+ _wandbruntime‚380  _runtime‚380+Ed¾Å&É¿M FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 358663747658 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚29717875778560 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚29357365454336 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚515 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 360510324224 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9436934 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 358471757234 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚16896418 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16917750 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4532974 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 374201263104 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 _runtime‚ 380.576933195 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚29357382493696 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚29718088347648 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚29357382493696 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6870138 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 360705853952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16896418 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16917750 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9458264 % _timestamp‚1.7883896248205566e+09 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4532977 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2337164 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7459484 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚29952256008192 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚34 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9436934 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 360705853952 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚29578318491648 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2337679 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2337679 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 358663747658 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚29717875778560 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚29357365454336 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚515 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 360510324224 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9436934 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 358471757234 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚16896418 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16917750 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4532974 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 374201263104 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 _runtime‚ 380.576933195 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚29357382493696 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚29718088347648 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚29357382493696 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6870138 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 360705853952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16896418 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16917750 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9458264 % _timestamp‚1.7883896248205566e+09 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4532977 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2337164 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7459484 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚29952256008192 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚34 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9436934 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 360705853952 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚29578318491648"8Ó5³0Ë+ _wandbruntime‚380  _runtime‚380P8E=RÌÌ /loss/FRt1/assignment_loss‚5.723087060798719 1loss/SRqqt2/assignment_loss‚2.226239433238556 1loss/SRqqt3/assignment_loss‚2.985598238503061 /loss/FRt3/assignment_loss‚4.870825483881194 0loss/FBt1/assignment_loss‚0.5313944518566132 0loss/FBt2/assignment_loss‚0.3450488069545135 1loss/FBt3/assignment_loss‚0.42143862495562645 0loss/FBt4/assignment_loss‚0.8406815826892853 0loss/SRqqt4/detection_loss‚2.777070203069913 /loss/FBt4/detection_loss‚2.5637216838923367 $loss/total_loss‚65.7393571819157 /loss/FRt2/detection_loss‚1.8919316462791749 /loss/FBt3/detection_loss‚2.8260164642333985 epoch‚0 trainer/global_step‚449 _step‚35 /loss/FRt2/assignment_loss‚4.122664150095334 1loss/SRqqt1/assignment_loss‚2.848192094905036 1loss/SRqqt2/detection_loss‚1.4155520068650895 _runtime‚ 380.577984977 /loss/FRt3/detection_loss‚3.0074531867586334 % _timestamp‚1.7883896248221717e+09 1loss/SRqqt1/detection_loss‚1.7270872990999904 /loss/FBt2/detection_loss‚0.6644647479131689 /loss/FRt4/assignment_loss‚6.175989151000977 2loss/SRqqt4/assignment_loss‚2.8120306811090243 0loss/SRqqt3/detection_loss‚3.123468403409167 /loss/FRt1/detection_loss‚2.4422426851629053 /loss/FRt4/detection_loss‚2.7768709616227585 .loss/FBt1/detection_loss‚6.620288133621216×ô¶ÔVÍÐ 0loss/FBt2/assignment_loss‚0.3450488069545135 1loss/FBt3/assignment_loss‚0.42143862495562645 0loss/FBt4/assignment_loss‚0.8406815826892853 0loss/SRqqt4/detection_loss‚2.777070203069913 /loss/FBt4/detection_loss‚2.5637216838923367 $loss/total_loss‚65.7393571819157 /loss/FRt2/detection_loss‚1.8919316462791749 /loss/FBt3/detection_loss‚2.8260164642333985 epoch‚0 trainer/global_step‚449 _step‚35 /loss/FRt2/assignment_loss‚4.122664150095334 1loss/SRqqt1/assignment_loss‚2.848192094905036 1loss/SRqqt2/detection_loss‚1.4155520068650895 _runtime‚ 380.577984977 /loss/FRt3/detection_loss‚3.0074531867586334 % _timestamp‚1.7883896248221717e+09 1loss/SRqqt1/detection_loss‚1.7270872990999904 /loss/FBt2/detection_loss‚0.6644647479131689 /loss/FRt4/assignment_loss‚6.175989151000977 2loss/SRqqt4/assignment_loss‚2.8120306811090243 0loss/SRqqt3/detection_loss‚3.123468403409167 /loss/FRt1/detection_loss‚2.4422426851629053 /loss/FRt4/detection_loss‚2.7768709616227585 .loss/FBt1/detection_loss‚6.620288133621216 /loss/FRt1/assignment_loss‚5.723087060798719 1loss/SRqqt2/assignment_loss‚2.226239433238556 1loss/SRqqt3/assignment_loss‚2.985598238503061 /loss/FRt3/assignment_loss‚4.870825483881194 0loss/FBt1/assignment_loss‚0.5313944518566132# !´ö’΂(:‡ ‚ÒâÔ«îÆu gpu.0.gpu‚28.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.978' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.5956 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0œ.œ )Ï‚(:ž ‚ÒâÔ±¾„y" proc.memory.rssMB‚ 2801.5234375 disk.sda3.in‚0 disk.drbd1777.out‚0$ memory_percent‚80.98001480102539' proc.memory.availableMB‚ 3116.234375% disk./.usageGB‚1042.3412590026855 disk.dm-1.out‚ 14.23828125 disk.sda.out‚ 14.23828125 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.rbd1.in‚0) proc.memory.percent‚17.09914207458496 disk.rbd3.out‚ 0.9873046875 disk.rbd1.out‚ 0.306640625 disk.rbd2.in‚0 disk.rbd2.out‚12.85400390625 disk.sda1.in‚0 disk.drbd1777.in‚0' disk.nvme0c0n1.out‚1842.81396484375 disk.sda2.in‚0" disk.dm-0.out‚1842.81396484375 disk.dm-1.in‚0 disk.sda.in‚0 cpu‚21.590011211058204 proc.cpu.threads‚17 network.sent‚595185 disk.nvme0n1.in‚ 39.7109375% disk.nvme0n1.out‚1842.81396484375) disk./.usagePercent‚69.95239647423966 disk.rbd3.in‚0 disk.dm-0.in‚ 39.7109375 disk.sda1.out‚0 network.recv‚197552 disk.sda3.out‚ 14.23828125 disk.nvme0c0n1.in‚ 39.7109375 disk.sda2.out‚0€ Iï‘Ђ(:† ‘ÒâÔºž¾u gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.145' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.829 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒ”$)Ñ‚(:ž ‘Òâԩߘx% disk.nvme0n1.out‚1869.78369140625 disk.sda2.out‚0 disk.sda3.in‚0$ memory_percent‚80.98044395446777) disk./.usagePercent‚69.95395530411565 disk.sda2.in‚0 disk.rbd2.out‚12.85400390625 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚ 39.7109375( proc.memory.availableMB‚ 3116.1640625 disk.sda1.out‚0 disk.rbd3.in‚0 proc.cpu.threads‚17$ disk./.usageGB‚1042.364486694336) proc.memory.percent‚17.09914207458496 cpu‚21.709641336326463 network.recv‚202666 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.dm-0.in‚ 39.7109375 disk.sda3.out‚ 15.08203125 network.sent‚644527 disk.rbd1.out‚ 0.306640625 disk.sda.out‚ 15.08203125" disk.dm-0.out‚1869.78369140625 disk.dm-1.in‚0" proc.memory.rssMB‚ 2801.5234375 disk.nvme0n1.in‚ 39.7109375 disk.sda.in‚0 disk.rbd0.out‚0.078125' disk.nvme0c0n1.out‚1869.78369140625 disk.rbd3.out‚ 0.9873046875 disk.dm-1.out‚ 15.08203125 disk.rbd1.in‚0 disk.rbd2.in‚0ß炲Ò‚(:’  ÒâÔ¡ª¿u gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.058' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.011599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0gX%Ó‚(:š  ÒâÔÿ̦x disk.rbd3.out‚ 1.0654296875 disk.rbd1.out‚ 0.306640625% disk./.usageGB‚1042.3391494750977$ disk.nvme0n1.out‚1973.2080078125 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd3.in‚0 network.sent‚651043 disk.dm-1.out‚ 15.3671875) proc.memory.percent‚17.09914207458496 disk.drbd1777.in‚0 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 15.3671875 disk.rbd1.in‚0 disk.rbd2.in‚0! disk.dm-0.out‚1973.2080078125) disk./.usagePercent‚69.95225490207324 disk.sda.out‚ 15.3671875 disk.nvme0c0n1.in‚ 39.7109375) proc.memory.availableMB‚ 3116.08203125 disk.sda.in‚0 disk.drbd1777.out‚0 disk.sda3.in‚0 disk.dm-1.in‚0 disk.rbd2.out‚14.03759765625 network.recv‚207711 disk.nvme0n1.in‚ 39.7109375& disk.nvme0c0n1.out‚1973.2080078125 disk.dm-0.in‚ 39.7109375$ memory_percent‚80.98094463348389 cpu‚21.821869439861956 disk.rbd0.out‚0.078125 disk.sda2.out‚0" proc.memory.rssMB‚ 2801.5234375Zs“Ë0Ô+ _wandbruntime‚421  _runtime‚421é ä>ÃÕ½ trainer/global_step‚499 $ _timestamp‚1.788389665953999e+09 _runtime‚ 421.710981822 _step‚36 & lr-AdamW/pg1‚0.0014198297825499195 & lr-AdamW/pg2‚0.0014198297825499195qµ–WÇÖÁ _step‚36 & lr-AdamW/pg1‚0.0014198297825499195 & lr-AdamW/pg2‚0.0014198297825499195 trainer/global_step‚499 $ _timestamp‚1.788389665953999e+09 _runtime‚ 421.710981822$õnp0×+ _wandbruntime‚421  _runtime‚421¤Wóû¿'عO =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚32544295303168 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚10463014 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚516 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚10484318 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32943937564672 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7616619 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚513 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2591463 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚32217998024798 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2591976 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8268964 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚31820615870464 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚18731978 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚33203882002944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8268966 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 397399193694 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 399854817280 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 399854817280 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8268964 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883896659547443e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5024643 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚32218207054374 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚31820598831104 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209029576 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5024640 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚33203608019456 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚32544278263808 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10463014 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32544278263808 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚32789092715008 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 397591183910 trainer/global_step‚499 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7616103 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 399659300864 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚32944150120448 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚32943937564672 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚32544295303168 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10484318 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚ 421.711089595 _step‚37 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5024640 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚33203608019456 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚32544278263808 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10463014 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322028032 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32544278263808 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚32789092715008 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 397591183910 trainer/global_step‚499 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7616103 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 399659300864 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚32944150120448 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚32943937564672 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚32544295303168 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10484318 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚ 421.711089595 _step‚37 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32943937564672 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322028032 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7616619 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚513 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2591463 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚32217998024798 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2591976 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚21304 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8268964 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚31820615870464 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚18731978 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚33203882002944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8268966 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 397399193694 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 399854817280 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 399854817280 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8268964 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883896659547443e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5024643 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MFDeviceStatsMonitor.on_train_batch_ƒ´˜TÕstart/inactive_split.large_pool.peak‚28 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚32218207054374 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243%üΠ0Ú+ _wandbruntime‚422  _runtime‚422º ð|È&ÛÂM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚8285484 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚32282600322106 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5034674 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚8285484 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 400457691648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 415615066624 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚18790702 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚33009987561472 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚18769370 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚32854618211328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10505216 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8285486 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7631240 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10505216 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 422.461848974 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚18790702 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10483886 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚33009987561472 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚32282391292322 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 398385067066 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8285486 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10483886 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2596566 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚486 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚33009774992384 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2597052 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚33269961404928 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚32854593307648 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚31884215255040 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚32609334340096 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚33270233277952 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 415368097280 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271873024 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚499 $ _timestamp‚1.788389666705213e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚489 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 400653221376 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚31884198215680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 398193076642 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚18769370 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5034677 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 _step‚38 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 400653221376 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚32609334340096 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0æ öDÌ&ÜÆM XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 400653221376 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚31884198215680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 398193076642 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚18769370 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5034677 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 _step‚38 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 400653221376 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚32609334340096 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 212569088 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚8285484 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚32282600322106 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5034674 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚8285484 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 400457691648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 415615066624 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚18790702 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚33009987561472 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚18769370 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚32854618211328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10505216 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8285486 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7631240 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 267950592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10505216 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 422.461848974 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚18790702 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10483886 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚33009987561472 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚32282391292322 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 398385067066 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 195529728 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8285486 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10483886 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2596566 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚486 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚33009774992384 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2597052 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚33269961404928 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚32854593307648 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚31884215255040 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚32609334340096 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚33270233277952 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 415368097280 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271873024 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191990424 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚499 $ _timestamp‚1.788389666705213e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚489 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507&†øŸe0Ý+ _wandbruntime‚422  _runtime‚422@l.MÞÇ trainer/global_step‚499 /loss/FRt1/assignment_loss‚4.131971888935443 /loss/FRt2/assignment_loss‚3.609103045825447 1loss/SRqqt3/assignment_loss‚2.461166653080263 0loss/FBt1/assignment_loss‚0.6350438843170801 0loss/FBt4/assignment_loss‚0.5439701934880077 /loss/FRt3/detection_loss‚2.6159863433776755 1loss/SRqqt1/detection_loss‚1.7020556022291597 2loss/SRqqt1/assignment_loss‚2.1889019624046657 1loss/SRqqt2/detection_loss‚1.1842118405018534 1loss/SRqqt4/detection_loss‚2.6301993666326298 1loss/SRqqt2/assignment_loss‚2.678479565892901 .loss/FRt2/detection_loss‚2.026332923743342 /loss/FRt4/detection_loss‚2.7127649503595688 0loss/SRqqt3/detection_loss‚2.269840865895368 % _timestamp‚1.7883896667070937e+09 _step‚39 -loss/FRt3/assignment_loss‚4.1345780812777 0loss/FBt2/assignment_loss‚0.3182998361802086 _runtime‚ 422.462899513 .loss/FRt1/detection_loss‚2.342470512869432 .loss/FBt2/detection_loss‚1.037790163983266 &loss/total_loss‚59.919869245142266 .loss/FBt1/detection_loss‚4.592726866404216 2loss/SRqqt4/assignment_loss‚2.7704953446107754 /loss/FBt3/assignment_loss‚0.657782772616407 epoch‚0 /loss/FRt4/assignment_loss‚5.892891220017975 .loss/FBt3/detection_loss‚4.052586975551787 .loss/FBt4/detection_loss‚2.730218384947096ËÑQßË 0loss/SRqqt3/detection_loss‚2.269840865895368 % _timestamp‚1.7883896667070937e+09 _step‚39 -loss/FRt3/assignment_loss‚4.1345780812777 0loss/FBt2/assignment_loss‚0.3182998361802086 _runtime‚ 422.462899513 .loss/FRt1/detection_loss‚2.342470512869432 .loss/FBt2/detection_loss‚1.037790163983266 &loss/total_loss‚59.919869245142266 .loss/FBt1/detection_loss‚4.592726866404216 2loss/SRqqt4/assignment_loss‚2.7704953446107754 /loss/FBt3/assignment_loss‚0.657782772616407 epoch‚0 /loss/FRt4/assignment_loss‚5.892891220017975 .loss/FBt3/detection_loss‚4.052586975551787 .loss/FBt4/detection_loss‚2.730218384947096 trainer/global_step‚499 /loss/FRt1/assignment_loss‚4.131971888935443 /loss/FRt2/assignment_loss‚3.609103045825447 1loss/SRqqt3/assignment_loss‚2.461166653080263 0loss/FBt1/assignment_loss‚0.6350438843170801 0loss/FBt4/assignment_loss‚0.5439701934880077 /loss/FRt3/detection_loss‚2.6159863433776755 1loss/SRqqt1/detection_loss‚1.7020556022291597 2loss/SRqqt1/assignment_loss‚2.1889019624046657 1loss/SRqqt2/detection_loss‚1.1842118405018534 1loss/SRqqt4/detection_loss‚2.6301993666326298 1loss/SRqqt2/assignment_loss‚2.678479565892901 .loss/FRt2/detection_loss‚2.026332923743342 /loss/FRt4/detection_loss‚2.7127649503595688'ÅŸ£à‚(:… ¯ÒâÔ‰ØÄu gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.26' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.452 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08¶ô/'á‚(:œ ¯ÒâÔìöÏx network.sent‚657853$ disk./.usageGB‚1042.339153289795 disk.drbd1777.out‚0 disk.sda3.out‚ 16.05078125) disk./.usagePercent‚69.95225515808076 disk.dm-1.in‚0 disk.rbd1.out‚ 0.306640625 disk.rbd2.in‚0 disk.rbd2.out‚15.04931640625$ disk.nvme0n1.out‚2071.5634765625 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda2.out‚0! disk.dm-0.out‚2071.5634765625( proc.memory.availableMB‚ 3117.3359375# proc.memory.rssMB‚ 2801.52734375 proc.cpu.threads‚17 disk.sda1.in‚0 disk.dm-0.in‚ 39.7109375 disk.rbd0.in‚0& disk.nvme0c0n1.out‚2071.5634765625 disk.rbd3.out‚ 1.0654296875 disk.sda3.in‚0$ memory_percent‚80.97329139709473 disk.nvme0n1.in‚ 39.7109375 disk.rbd0.out‚0.078125) proc.memory.percent‚17.09916591644287 cpu‚21.928764522068793 disk.dm-1.out‚ 16.05078125 disk.sda.out‚ 16.05078125 disk.sda2.in‚0 network.recv‚212654 disk.nvme0c0n1.in‚ 39.7109375 disk.rbd3.in‚0öÐÆ‘â‚(:† ¾ÒâÔÐÛ¿u gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.385' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.877 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ì¦\ã‚(:Ž ¾ÒâÔØæýw disk.nvme0n1.in‚ 39.7109375# proc.memory.rssMB‚ 2801.52734375 disk.sda.out‚16.375 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.dm-1.out‚16.375 disk.rbd1.out‚ 0.306640625$ memory_percent‚80.97209930419922$ disk./.usageGB‚1042.337314605713 disk.sda.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.1591796875 disk.sda3.out‚16.375 disk.rbd1.in‚0& proc.memory.availableMB‚ 3117.53125 proc.cpu.threads‚17" disk.dm-0.out‚2072.47900390625 disk.dm-1.in‚0 network.recv‚217669) disk./.usagePercent‚69.95213176244926 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda3.in‚0 disk.rbd2.out‚16.30322265625% disk.nvme0n1.out‚2072.47900390625 disk.rbd2.in‚0 disk.dm-0.in‚ 39.7109375) proc.memory.percent‚17.09916591644287 network.sent‚707073 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 39.7109375' disk.nvme0c0n1.out‚2072.47900390625 cpu‚22.029503115811224åÁÆ0ä+ _wandbruntime‚463  _runtime‚463¡è¸]Äå¾ trainer/global_step‚549 % _timestamp‚1.7883897079196997e+09 _runtime‚ 463.675748779 _step‚40 & lr-AdamW/pg1‚0.0015620972958314747 & lr-AdamW/pg2‚0.00156209729583147477Ãv»Èæ _runtime‚ 463.675748779 _step‚40 & lr-AdamW/pg1‚0.0015620972958314747 & lr-AdamW/pg2‚0.0015620972958314747 trainer/global_step‚549 % _timestamp‚1.7883897079196997e+09(JÜM£0ç+ _wandbruntime‚463  _runtime‚463¹~ÒÜè¹O [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚35796230110208 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚520 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5526340 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚36521546396160 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚36235836778496 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5526343 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 439606668288 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚36235836778496 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚35796247149568 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 437312503318 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9094966 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 439802184704 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267692032 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35796230110208 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8375849 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚36065367531008 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚35000585099264 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 437120513102 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 270042112 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11531270 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 % _timestamp‚1.7883897079208932e+09 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚35000568059904 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 455936076800 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 439802184704 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242788352 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚20604930 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2850026 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚35796247149568 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11531270 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 _step‚41 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36521278704128 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2849509 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚36065342627328 trainer/global_step‚549 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9094964 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 456178865152ú×èÃ'é½O OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚21304 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 455936076800 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494339229 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 439802184704 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242788352 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚20604930 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2850026 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958634389 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚35796247149568 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11531270 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 _step‚41 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36521278704128 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2849509 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚36065342627328 trainer/global_step‚549 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9094964 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 456178865152 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚35796230110208 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚520 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5526340 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191990216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚36521546396160 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚36235836778496 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 497872384 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5526343 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 439606668288 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚36235836778496 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚35796247149568 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚26287 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 437312503318 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9094966 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 439802184704 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267692032 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35796230110208 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8375849 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚36065367531008 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚35000585099264 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 437120513102 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 270042112 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚26287 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11531270 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 195516416 % _timestamp‚1.7883897079208932e+09 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 212555776 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚35000568059904 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28)¸µÑ0ê+ _wandbruntime‚464  _runtime‚464XN>È&ëÂM E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚36130893027328 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9111484 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 249066496 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35861286186496 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11530838 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 $ _timestamp‚1.788389708680646e+09 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9111484 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 440600588800 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8391012 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 438106386474 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚35064167444480 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚20642322 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9111486 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 440405059072 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚495 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2854638 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 440405059072 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9111486 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2855133 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚36130868123648 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 440600588800 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚20663654 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚35861269147136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273970176#Ê°?"ìÆM I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9111484 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 249066496 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35861286186496 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 195529728 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11530838 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 $ _timestamp‚1.788389708680646e+09 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9111484 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 440600588800 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8391012 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 438106386474 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚35064167444480 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚20642322 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9111486 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 440405059072 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚495 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2854638 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 440405059072 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9111486 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2855133 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚36130868123648 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 440600588800 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚20663654 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494339229 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚35861269147136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273970176 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚36130893027328 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚21330 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0*ƒÏX0í+ _wandbruntime‚464  _runtime‚464׬»QîË .loss/FRt1/detection_loss‚2.323266585047046 2loss/SRqqt4/assignment_loss‚2.9749643599733395 1loss/SRqqt3/detection_loss‚2.1670275803407035 /loss/FBt4/detection_loss‚2.6816277160094333 _step‚43 /loss/FRt3/detection_loss‚2.5035236347012404 0loss/SRqqt1/detection_loss‚1.791461483039687 % _timestamp‚1.7883897086823153e+09 -loss/FBt3/assignment_loss‚0.7310009775064 0loss/FBt4/assignment_loss‚0.7687689047201485 /loss/FRt2/detection_loss‚2.0005937665700912 /loss/FRt1/assignment_loss‚4.242697160070141 /loss/FRt3/assignment_loss‚4.480025753742311 /loss/FRt4/assignment_loss‚5.881828065576224 2loss/SRqqt1/assignment_loss‚2.1179529387602765 1loss/SRqqt2/assignment_loss‚2.252346569059679 0loss/FBt1/assignment_loss‚0.4425320892284314 2loss/SRqqt3/assignment_loss‚2.1027551317214965 1loss/SRqqt4/detection_loss‚2.9104459666191262 /loss/FBt2/detection_loss‚0.8049446227928174 epoch‚0 trainer/global_step‚549 1loss/FBt2/assignment_loss‚0.34887028387392416 .loss/FRt4/detection_loss‚2.512838008074925 %loss/total_loss‚55.91656554848165 _runtime‚ 464.438072382 1loss/SRqqt2/detection_loss‚1.2608437102112995 .loss/FBt1/detection_loss‚3.562767873207728 /loss/FRt2/assignment_loss‚2.956890913071456 /loss/FBt3/detection_loss‚2.0965914545637188Ϙ„…UïÏ /loss/FBt4/detection_loss‚2.6816277160094333 _step‚43 /loss/FRt3/detection_loss‚2.5035236347012404 0loss/SRqqt1/detection_loss‚1.791461483039687 % _timestamp‚1.7883897086823153e+09 -loss/FBt3/assignment_loss‚0.7310009775064 0loss/FBt4/assignment_loss‚0.7687689047201485 /loss/FRt2/detection_loss‚2.0005937665700912 /loss/FRt1/assignment_loss‚4.242697160070141 /loss/FRt3/assignment_loss‚4.480025753742311 /loss/FRt4/assignment_loss‚5.881828065576224 2loss/SRqqt1/assignment_loss‚2.1179529387602765 1loss/SRqqt2/assignment_loss‚2.252346569059679 0loss/FBt1/assignment_loss‚0.4425320892284314 2loss/SRqqt3/assignment_loss‚2.1027551317214965 1loss/SRqqt4/detection_loss‚2.9104459666191262 /loss/FBt2/detection_loss‚0.8049446227928174 epoch‚0 trainer/global_step‚549 1loss/FBt2/assignment_loss‚0.34887028387392416 .loss/FRt4/detection_loss‚2.512838008074925 %loss/total_loss‚55.91656554848165 _runtime‚ 464.438072382 1loss/SRqqt2/detection_loss‚1.2608437102112995 .loss/FBt1/detection_loss‚3.562767873207728 /loss/FRt2/assignment_loss‚2.956890913071456 /loss/FBt3/detection_loss‚2.0965914545637188 .loss/FRt1/detection_loss‚2.323266585047046 2loss/SRqqt4/assignment_loss‚2.9749643599733395 1loss/SRqqt3/detection_loss‚2.1670275803407035+‚#ñ0’ð‚(:‡ ÍÒâÔ“ÀÓu gpu.0.gpu‚55.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.347' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.0694 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0”`)<&ñ‚(:› ÍÒâÔâÿàw$ disk./.usageGB‚1042.337989807129 disk.rbd3.in‚0 disk.dm-1.out‚ 17.28515625 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 39.7109375 disk.sda2.in‚0 disk.rbd2.out‚16.31103515625% disk.nvme0n1.out‚2170.86181640625 disk.sda.out‚ 17.28515625" disk.dm-0.out‚2170.86181640625 disk.drbd1777.in‚0 disk.dm-1.in‚0) proc.memory.percent‚17.09918975830078 network.recv‚221777' disk.nvme0c0n1.out‚2170.86181640625 disk.rbd3.out‚ 1.1591796875 disk.sda3.in‚0 disk.rbd1.in‚0 proc.cpu.threads‚17 disk.rbd1.out‚ 0.306640625 disk.sda1.out‚0 disk.rbd0.in‚0 disk.dm-0.in‚ 39.7109375 disk.sda3.out‚ 17.28515625 disk.rbd2.in‚0( proc.memory.availableMB‚ 3117.5390625 network.sent‚712864 disk.nvme0n1.in‚ 39.7109375 disk.drbd1777.out‚0 disk.sda2.out‚0# memory_percent‚80.9720516204834 proc.memory.rssMB‚ 2801.53125) disk./.usagePercent‚69.95217707578281 disk.sda.in‚0 cpu‚22.125340078747573¶#‹º’ò‚(:‡ ÜÒâÔéðÂu gpu.0.gpu‚55.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚138.993' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.7986 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0R7â$ó‚(:™ ÜÒâÔêÇŠx) proc.memory.percent‚17.09918975830078 cpu‚22.216213799825685 disk.dm-1.in‚0 disk.rbd2.in‚0$ disk.nvme0n1.out‚2270.7041015625 disk.sda.in‚0 disk.sda2.in‚0& proc.memory.availableMB‚ 3117.40625 disk.rbd1.in‚0 disk.rbd2.out‚17.50244140625 disk.drbd1777.in‚0& disk.nvme0c0n1.out‚2270.7041015625 proc.cpu.threads‚17 network.sent‚762958 network.recv‚227572 disk.nvme0n1.in‚ 39.7109375 disk.sda.out‚ 17.71484375 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚ 39.7109375 disk.dm-1.out‚ 17.71484375 disk.rbd1.out‚ 0.306640625 disk.dm-0.in‚ 39.7109375 disk.rbd0.in‚0) disk./.usagePercent‚69.95215045099926 disk.sda3.out‚ 17.71484375$ memory_percent‚80.97286224365234% disk./.usageGB‚1042.3375930786133 disk.rbd3.in‚0 disk.rbd3.out‚ 1.22216796875! disk.dm-0.out‚2270.7041015625 disk.sda3.in‚0 disk.sda1.in‚0 disk.sda2.out‚0 disk.sda1.out‚0 disk.rbd0.out‚0.078125 proc.memory.rssMB‚ 2801.53125‰°3!’ô‚(:‡ ëÒâÔÛ›Êu gpu.0.gpu‚57.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.256' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.6512 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0†*N õ‚(:• ëÒâÔºóÓw% disk./.usageGB‚1042.3406448364258 disk.sda.in‚0 disk.sda2.out‚0 disk.rbd1.in‚0 disk.rbd2.out‚18.51416015625 disk.nvme0n1.in‚ 39.7109375) proc.memory.percent‚17.09921360015869 disk.dm-1.in‚0 disk.rbd2.in‚0# proc.memory.rssMB‚ 2801.53515625 cpu‚22.30218004587567 disk.drbd1777.out‚0) disk./.usagePercent‚69.95235525702664 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.22216796875) proc.memory.availableMB‚ 3116.81640625 disk.sda.out‚ 18.3046875 disk.rbd0.out‚0.078125$ disk.nvme0c0n1.out‚ 2274.44921875 disk.sda2.in‚0 network.sent‚769868 disk.sda1.in‚0 disk.dm-0.in‚ 39.7109375 disk.dm-0.out‚ 2274.44921875 disk.sda3.in‚0 disk.dm-1.out‚ 18.3046875 disk.nvme0c0n1.in‚ 39.7109375 disk.sda3.out‚ 18.3046875" disk.nvme0n1.out‚ 2274.44921875$ memory_percent‚80.97646236419678 disk.sda1.out‚0 disk.rbd1.out‚ 0.306640625 proc.cpu.threads‚17 network.recv‚232663æÏÓ0ö+ _wandbruntime‚505  _runtime‚505û‚Ã!Ã÷½ & lr-AdamW/pg2‚0.0017043648091130297 trainer/global_step‚599 $ _timestamp‚1.788389749757736e+09 _runtime‚ 505.513748399 _step‚44 & lr-AdamW/pg1‚0.0017043648091130297¸/ÇøÁ & lr-AdamW/pg1‚0.0017043648091130297 & lr-AdamW/pg2‚0.0017043648091130297 trainer/global_step‚599 $ _timestamp‚1.788389749757736e+09 _runtime‚ 505.513748399 _step‚44,6bûë0ù+ _wandbruntime‚505  _runtime‚505Ÿ²Qª¾'ú¸O $ _timestamp‚1.788389749758892e+09 _runtime‚ 505.515165179 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 212555776 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39527735992320 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚38657379121214 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39527948548096 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚486 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 257468416 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚39048181956608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 479554035712 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9136351 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚39048198995968 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 479749552128 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚38657588150790 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39527735992320 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚38657379121214 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 497872384 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39527948548096 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚26034 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚486 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 257468416 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚21306 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚26034 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚39048181956608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 195516416 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚21306 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 479554035712 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9136351-••HÆ0ü+ _wandbruntime‚506  _runtime‚506y$Ã,e ýÃM N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39593785989120 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22515274 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 480547956224 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22515274 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 480352426496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚38244153712640 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 506.267867259 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚22536606 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚12577790 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚38721772388738 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚39113238032896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6038074 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883897505111926e+09 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6038077 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12577790 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9150937 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚39113238032896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚39593573420032 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12599120 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚38721981418522 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 477827705882 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 498216613888 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 480547956224 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9937484 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39407142939648 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12599120 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚39593785989120 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9937486 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9937486 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271873024 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9937484 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39407167843328 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚39905359553536 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22536606 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3113365 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332›ù.Í&þÇM TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚39905359553536 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22536606 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚26034 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3113365 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚21332 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322028032 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39593785989120 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22515274 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 480547956224 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209029784 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22515274 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 497872384 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 480352426496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚38244153712640 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 506.267867259 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚22536606 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚12577790 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚38721772388738 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322028032 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚39113238032896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6038074 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883897505111926e+09 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6038077 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚26287 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12577790 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9150937 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚39113238032896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚39593573420032 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12599120 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚21330 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚38721981418522 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 477827705882 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 498216613888 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 480547956224 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9937484 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39407142939648 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12599120 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚26287 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚39593785989120 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 497872384 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9937486 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246969344 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9937486 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958634389 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 212569088 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271873024 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9937484 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39407167843328 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0.:ˆÃ}0ÿ+ _wandbruntime‚506  _runtime‚506S²­CP€Ê 0loss/FBt3/assignment_loss‚0.4285241284374836 epoch‚0 0loss/FBt4/assignment_loss‚0.7097229573620325 1loss/SRqqt4/detection_loss‚2.7123167037017764 .loss/FBt4/detection_loss‚3.589088197797537 /loss/FRt1/assignment_loss‚4.271908614879999 0loss/FRt2/assignment_loss‚2.6975243389606476 0loss/FBt1/assignment_loss‚0.7032802127301693 1loss/FBt2/assignment_loss‚0.23317619294235736 /loss/FBt2/detection_loss‚0.7248069958970744 /loss/FRt3/assignment_loss‚4.263512386364884 1loss/SRqqt1/assignment_loss‚1.865655275205007 _runtime‚ 506.268962092 1loss/SRqqt2/assignment_loss‚2.132867583497004 2loss/SRqqt4/assignment_loss‚2.7337996458250378 1loss/SRqqt3/detection_loss‚1.9741359455328362 .loss/FBt3/detection_loss‚2.834951387984412 trainer/global_step‚599 .loss/FRt1/detection_loss‚2.695593692171268 0loss/SRqqt2/detection_loss‚1.231548810659936 /loss/FBt1/detection_loss‚2.6828167498111726 %loss/total_loss‚55.27826658270526 % _timestamp‚1.7883897505131598e+09 _step‚47 /loss/FRt4/assignment_loss‚5.377558037638664 /loss/FRt2/detection_loss‚1.8888886566547787 2loss/SRqqt3/assignment_loss‚2.4146055643465005 .loss/FRt3/detection_loss‚2.382701673534479 .loss/FRt4/detection_loss‚2.729128211736679 /loss/SRqqt1/detection_loss‚2.00015461903352Wq2TÎ .loss/FRt3/detection_loss‚2.382701673534479 .loss/FRt4/detection_loss‚2.729128211736679 /loss/SRqqt1/detection_loss‚2.00015461903352 0loss/FBt3/assignment_loss‚0.4285241284374836 epoch‚0 0loss/FBt4/assignment_loss‚0.7097229573620325 1loss/SRqqt4/detection_loss‚2.7123167037017764 .loss/FBt4/detection_loss‚3.589088197797537 /loss/FRt1/assignment_loss‚4.271908614879999 0loss/FRt2/assignment_loss‚2.6975243389606476 0loss/FBt1/assignment_loss‚0.7032802127301693 1loss/FBt2/assignment_loss‚0.23317619294235736 /loss/FBt2/detection_loss‚0.7248069958970744 /loss/FRt3/assignment_loss‚4.263512386364884 1loss/SRqqt1/assignment_loss‚1.865655275205007 _runtime‚ 506.268962092 1loss/SRqqt2/assignment_loss‚2.132867583497004 2loss/SRqqt4/assignment_loss‚2.7337996458250378 1loss/SRqqt3/detection_loss‚1.9741359455328362 .loss/FBt3/detection_loss‚2.834951387984412 trainer/global_step‚599 .loss/FRt1/detection_loss‚2.695593692171268 0loss/SRqqt2/detection_loss‚1.231548810659936 /loss/FBt1/detection_loss‚2.6828167498111726 %loss/total_loss‚55.27826658270526 % _timestamp‚1.7883897505131598e+09 _step‚47 /loss/FRt4/assignment_loss‚5.377558037638664 /loss/FRt2/detection_loss‚1.8888886566547787 2loss/SRqqt3/assignment_loss‚2.4146055643465005/ص©’‚‚(:‡ úÒâÔì•øw gpu.0.gpu‚11.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚35.0 gpu.0.powerWatts‚154.331' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.8662 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0™{Âã$ƒ‚(:™ úÒâÔÑ“¼x disk.sda1.in‚0) disk./.usagePercent‚69.95238008975747 disk.sda2.out‚0 disk.dm-0.in‚ 39.7109375 disk.sda3.out‚ 18.7421875 disk.drbd1777.out‚0# proc.memory.rssMB‚ 2801.53515625) proc.memory.percent‚17.09921360015869 cpu‚22.384261288781286! disk.dm-0.out‚2373.8681640625 disk.sda3.in‚0 disk.dm-1.in‚0 disk.rbd2.out‚19.65478515625 disk.sda.in‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚ 39.7109375 disk.rbd0.in‚0 network.sent‚776025 disk.sda2.in‚0 disk.rbd3.out‚ 1.28515625 disk.dm-1.out‚ 18.7421875 disk.rbd1.out‚ 0.306640625 network.recv‚237557% disk./.usageGB‚1042.3410148620605 disk.sda.out‚ 18.7421875$ memory_percent‚80.97636699676514 disk.rbd3.in‚0 disk.rbd1.in‚0$ disk.nvme0n1.out‚2373.8681640625 disk.sda1.out‚0) proc.memory.availableMB‚ 3116.83203125& disk.nvme0c0n1.out‚2373.8681640625 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 proc.cpu.threads‚17 disk.nvme0c0n1.in‚ 39.7109375ÝÏØiš„‚(: ‰ÓâÔÎóÀu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.58' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.715999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0se’b*…‚(:Ÿ ‰ÓâÔ– œx% disk.nvme0n1.out‚2476.48876953125* proc.memory.percent‚17.562413215637207 proc.cpu.threads‚17 network.recv‚242510% disk./.usageGB‚1042.3411293029785 disk.dm-0.in‚ 39.7109375 disk.dm-1.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 2877.42578125' disk.nvme0c0n1.out‚2476.48876953125 network.sent‚825164 disk.nvme0n1.in‚ 39.7109375 disk.sda.in‚0 disk.nvme0c0n1.in‚ 39.7109375 disk.rbd3.out‚ 1.28515625" disk.dm-0.out‚2476.48876953125 disk.sda3.out‚ 22.18359375 disk.dm-1.out‚ 22.18359375 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.sda3.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.306640625$ memory_percent‚81.43980503082275) proc.memory.availableMB‚ 3040.90234375 cpu‚22.59081013638218 disk.rbd0.out‚0.078125) disk./.usagePercent‚69.95238776998349 disk.rbd2.in‚0 disk.sda.out‚ 22.18359375 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.rbd2.out‚20.66650390625 disk.sda1.in‚0 disk.sda2.out‚0e­:¾’†‚(:‡ ˜ÓâÔÿÊv gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚163.471' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6942 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0>£ì‡‚(:‘ ˜ÓâÔƒÓów# proc.memory.rssMB‚ 2877.70703125 proc.cpu.threads‚17) disk./.usagePercent‚69.95239161009651 disk.sda.in‚0 disk.sda2.out‚0$ memory_percent‚81.44147396087646 network.sent‚831540 disk.dm-0.in‚ 39.7109375 disk.dm-0.out‚ 2479.8046875 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚ 39.7109375 disk.rbd3.out‚ 1.28515625 network.recv‚247389% disk./.usageGB‚1042.3411865234375 disk.sda3.out‚ 22.67578125 disk.sda1.in‚0 disk.sda2.in‚0) proc.memory.percent‚17.56412982940674 cpu‚22.77610824190283 disk.rbd1.out‚ 0.306640625 disk.rbd2.in‚0 disk.rbd2.out‚ 20.8623046875! disk.nvme0n1.out‚ 2479.8046875# disk.nvme0c0n1.out‚ 2479.8046875 disk.dm-1.out‚ 22.67578125 disk.rbd1.in‚0 disk.sda.out‚ 22.67578125 disk.rbd0.in‚0 disk.drbd1777.in‚0) proc.memory.availableMB‚ 3040.62890625 disk.sda3.in‚0 disk.dm-1.in‚0 disk.nvme0n1.in‚ 39.7109375 disk.sda1.out‚0 disk.rbd3.in‚0l³”bˆ‚(:’ §ÓâÔ–éÂu gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚158.633' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.726599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Á¨ý»‰‚(:‹ §ÓâÔ£ñów proc.memory.rssMB‚ 2895.1875 cpu‚22.96698156558534 proc.cpu.threads‚17% disk./.usageGB‚1042.3411865234375 disk.sda1.out‚0 disk.rbd1.out‚ 0.306640625 disk.rbd2.out‚ 21.8740234375 disk.sda.in‚0 disk.rbd1.in‚0 network.recv‚252165 disk.drbd1777.in‚0 disk.dm-1.out‚ 23.234375 disk.rbd2.in‚0$ memory_percent‚81.53598308563232 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.dm-0.in‚ 39.7109375 network.sent‚838094) disk./.usagePercent‚69.95239161009651 disk.nvme0n1.in‚ 39.7109375" disk.nvme0n1.out‚ 2578.14453125$ disk.nvme0c0n1.out‚ 2578.14453125 disk.sda2.out‚0 disk.sda3.out‚ 23.234375* proc.memory.percent‚17.670822143554688 disk.sda.out‚ 23.234375 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.28515625 disk.dm-1.in‚0 disk.sda1.in‚0 disk.nvme0c0n1.in‚ 39.7109375 disk.drbd1777.out‚0 disk.dm-0.out‚ 2578.14453125 disk.sda3.in‚0) proc.memory.availableMB‚ 3025.14453125ŽÚ9±;Šb6 jet/accuracy_1_of_5"trainer/global_step2:JPñ†»J‹bE "Purity/1FRt1SRqqt3FBt/event_purity"trainer/global_step2:JPÆÎÿ JŒbE "Purity/4FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPèHbC Purity/0FRt2SRqqt3FBt/FBt_purity"trainer/global_step2:JPõŒ–NŽbI &Purity/0FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPD×sÕNbI &Purity/3FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPA?JbE "Purity/0FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¿>ZøJ‘bE "Purity/0FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP§Í)XJ’bE "Purity/3FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPrˆÉJ“bE "Purity/*FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP̨Z~H”bC Purity/1FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPPÆ^äN•bI &Purity/*FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPµ˜FßJ–bE "Purity/0FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPÉ[×@N—bI &Purity/0FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPë!\“H˜bC Purity/1FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JP.ÔM‚J™bE "Purity/1FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPÔ—ÒãJšbE "Purity/2FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP€ÖòÇN›bI &Purity/2FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPmÉdHœbC Purity/0FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JPÅbãÎNbI &Purity/*FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP_€;žb6 jet/accuracy_1_of_4"trainer/global_step2:JPðš­‰JŸbE "Purity/*FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPñoãÿJ bE "Purity/*FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPêÈ“ˆJ¡bE "Purity/0FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPaç^H¢bC Purity/0FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPR©mÚJ£bE "Purity/1FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPqúLéN¤bI &Purity/1FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPÀ€H¥bC Purity/2FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JPà)iN¦bI &Purity/3FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP]0ËU;§b6 jet/accuracy_4_of_4"trainer/global_step2:JPÁ¥üN¨bI &Purity/2FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPp3{eN©bI &Purity/4FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPÕeªCHªbC Purity/*FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JP+û9H«bC Purity/*FRt2SRqqt3FBt/FBt_purity"trainer/global_step2:JPXù0;¬b6 loss/val_total_loss"trainer/global_step2:JP|´ OH­bC Purity/1FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPvLJ®bE "Purity/2FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPû9¸2@¯b; particle/accuracy_2_of_3"trainer/global_step2:JP²úÌyH°bC Purity/2FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JP#r:ÞH±bC Purity/1FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JPvP!H²bC Purity/*FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPõ¢J³bE "Purity/0FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPý˜J´bE "Purity/0FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPßÄ‹@NµbI &Purity/3FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÛ@ØN¶bI &Purity/4FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPwEN·bI &Purity/4FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPÑTë^J¸bE "Purity/1FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPèŠN¹bI &Purity/0FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP¹G”HºbC Purity/1FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPºõk•H»bC Purity/3FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP@|Z@¼b; particle/accuracy_2_of_7"trainer/global_step2:JP»§J”J½bE "Purity/0FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP¹=Ï8J¾bE "Purity/2FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPšt9H¿bC Purity/*FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPý#†HÀbC Purity/2FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPú;JÁbE "Purity/2FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP6Of’JÂbE "Purity/2FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPZÛHâHÃbC Purity/*FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP„gÛ„HÄbC Purity/*FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP0@@NÅbI &Purity/*FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPÿ_HÆbC Purity/0FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP¤ ÅlHÇbC Purity/2FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP~úA·NÈbI &Purity/3FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPg†•NÉbI &Purity/4FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP`ÒÿðNÊbI &Purity/4FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPrê¨NËbI &Purity/3FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP¾þ-;Ìb6 jet/accuracy_1_of_2"trainer/global_step2:JPA@¦ðJÍbE "Purity/*FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP -¦HÎbC Purity/0FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP•n‘HÏbC Purity/1FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPKUÿô;Ðb6 jet/accuracy_2_of_7"trainer/global_step2:JP¸Ý6HÑbC Purity/1FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JP­+/…JÒbE "Purity/*FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPŽVnJÓbE "Purity/*FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP "THÔbC Purity/3FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPp÷`JÕbE "Purity/3FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPé܇äJÖbE "Purity/1FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP^Ü®ÌJ×bE "Purity/0FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP¼¤·JØbE "Purity/1FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPPm÷pJÙbE "Purity/3FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¹UƦNÚbI &Purity/4FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP“ပJÛbE "Purity/4FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÛûHÜbC Purity/1FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPåÝIHÝbC Purity/1FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPWá2NÞbI &Purity/1FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPµ›*BNßbI &Purity/2FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP›‚]HàbC Purity/2FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPDw¤HábC Purity/2FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPÜÂ~7HâbC Purity/*FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JPÓ±e¸NãbI &Purity/*FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPºLù]HäbC Purity/1FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPø¦Ý¨NåbI &Purity/2FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP™Â™%<æb7 particle/specificity"trainer/global_step2:JPñ×cHçbC Purity/3FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPJ«z/HèbC Purity/*FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JP‰ • ébE "Puriªî??ty/3FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPztHJêbE "Purity/0FRt0SRqqt3FBt/event_purity"trainer/global_step2:JP›$'NëbI &Purity/0FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP‘šÞ#JìbE "Purity/2FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP$ïoHíbC Purity/3FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP°kÌJîbE "Purity/*FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPH¬<ÇJïbE "Purity/2FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP*ä­ÅJðbE "Purity/1FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPèƒEJñbE "Purity/4FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP”×2RJòbE "Purity/*FRt0SRqqt3FBt/event_purity"trainer/global_step2:JP8ÊuNóbI &Purity/2FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPã8hûJôbE "Purity/2FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPHê½$JõbE "Purity/*FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPˆ;ŒeHöbC Purity/4FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP#½?öH÷bC Purity/2FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP˜åÐNøbI &Purity/2FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP¿-ÝJùbE "Purity/0FRt0SRqqt4FBt/event_purity"trainer/global_step2:JP«y\JúbE "Purity/2FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPý4ÔHûbC Purity/1FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JPmqÌ¿JübE "Purity/*FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP° õOHýbC Purity/1FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPÍ2‘•HþbC Purity/1FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP«í ™HÿbC Purity/2FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JP¿hNÔJ€bE "Purity/3FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPÊ4JwNbI &Purity/3FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPå-J‚bE "Purity/2FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP#²Ù8@ƒb; particle/accuracy_2_of_5"trainer/global_step2:JP6z+@„b; particle/accuracy_5_of_7"trainer/global_step2:JPA5—N…bI &Purity/1FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP­Ë—«N†bI &Purity/1FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPŽ¸¦H‡bC Purity/2FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPͺ³#JˆbE "Purity/2FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPìPçN‰bI &Purity/3FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP½z'FHŠbC Purity/0FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPo×!;‹b6 jet/accuracy_4_of_6"trainer/global_step2:JPiÛJsNŒbI &Purity/*FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP|ß—ÕHbC Purity/*FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP·VNŽbI &Purity/*FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPUÍ‹HbC Purity/*FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP„I”NHbC Purity/*FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JPY6€nN‘bI &Purity/0FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP«A³N’bI &Purity/4FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP¨06N“bI &Purity/*FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPŸe}ÉH”bC Purity/*FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPÒWÙH•bC Purity/0FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP= ‹N–bI &Purity/0FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPpŒWÈJ—bE "Purity/2FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPºÀ~`H˜bC Purity/*FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP;@ÈH™bC Purity/2FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP¶îä~JšbE "Purity/*FRt*SRqqt0FBt/event_purity"trainer/global_step2:JP¸Üº²N›bI &Purity/2FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPÏΘ™HœbC Purity/2FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP1cMcNbI &Purity/3FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP‹Ž*fNžbI &Purity/4FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP¨GK~JŸbE "Purity/4FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPìdª¥H bC Purity/*FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPÃoXJ¡bE "Purity/0FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP¯!}H¢bC Purity/0FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP‡W²ÆN£bI &Purity/0FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP¨Ý~ON¤bI &Purity/1FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP\óÈH¥bC Purity/3FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPÏqCeJ¦bE "Purity/3FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPŸ?šëH§bC Purity/1FRt1SRqqt4FBt/FRt_purity"trainer/global_step2:JP¬K¹üJ¨bE "Purity/*FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP:ãæsN©bI &Purity/0FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP¤IKäNªbI &Purity/1FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPq˜i´;«b6 jet/accuracy_3_of_4"trainer/global_step2:JP6mJ¬bE "Purity/1FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP&D!\J­bE "Purity/0FRt2SRqqt3FBt/event_purity"trainer/global_step2:JPiZö$N®bI &Purity/0FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPœ `QN¯bI &Purity/0FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP´b ­H°bC Purity/1FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP`œ‘kJ±bE "Purity/1FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPó'êH²bC Purity/1FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP¿›_ŒJ³bE "Purity/2FRt0SRqqt0FBt/event_purity"trainer/global_step2:JP_üDN´bI &Purity/2FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP €G HµbC Purity/3FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPÚßôJ¶bE "Purity/*FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPóJ·bE "Purity/*FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP7Æ›ÕJ¸bE "Purity/0FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP™ì N¹bI &Purity/1FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPÖø'HºbC Purity/3FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPÔ¢¡;»b6 jet/accuracy_1_of_3"trainer/global_step2:JP¯Tm•H¼bC Purity/*FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPt `:H½bC Purity/2FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPˆUAJ¾bE "Purity/1FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPK"—k;¿b6 jet/accuracy_5_of_5"trainer/global_step2:JPÃälJÀbE "Purity/1FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPk×Ð¥JÁbE "Purity/0FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP`,ܹHÂbC Purity/1FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JPÀª½SNÃbI &Purity/*FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPàPNIJÄbE "Purity/*FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPÐ#€JÅbE "Purity/0FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPWJLJÆbE "Purity/0FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPŒÿNÇbI &Purity/1FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPP‡úNÈbI &Purity/2FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP|ùåJÉbE "Purity/2FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPï BcNÊbI &Purity/3FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPå_ÝNËbI &Purity/*FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPiþJÌbE "Purity/*FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¯ŽeHÍbC Purity/3FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPRQXNÎbI &Purity/3FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP$ñfÂNÏbI &Purity/3FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPôÙ;Ðb6 jet/accuracy_4_of_5"trainer/global_step2:JPYƒîÞHÑbC Purity/*FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JPfó£ºJÒbE "Purity/1FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPNÁÉNÓbI &Purity/*FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP•Ô;JÔbE "Purity/2FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPÎ4ˆ NÕbI &Purity/4FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPõ²é1@Öb; particle/accuracy_7_of_7"trainer/global_step2:JP8£ãhH×bC Purity/*FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JPN¨AHØbC Purity/1FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPdQšNÙbI &Purity/0FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP ]TNÚbI &Purity/1FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPŽSNÉNÛbI &Purity/2FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPz÷¦»NÜbI &Purity/4FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPÚϾ;Ýb6 jet/accuracy_1_of_7"trainer/global_step2:JP-òz @Þb; particle/accuracy_4_of_7"trainer/global_step2:JP¡‰GŸJßbE "Purity/0FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPѱ tHàbC Purity/4FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP©ƒ˜vJábE "Purity/1FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPh›NâbI &Purity/3FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPéýùõHãbC Purity/*FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP¿«YJäbE "Purity/*FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPåE'óJåbE "Purity/0FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPБ+£NæbI &Purity/1FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPÆl,‰JçbE "Purity/1FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP,&Çw@èb; particle/accuracy_2_of_6"trainer/global_step2:JPlÛjQJébE "Purity/*FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¥JêbE "Purity/*FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPýé8NNëbI &Purity/*FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPÇ} JìbE "Purity/1FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP]S,HíbC Purity/3FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP—JîbE "Purity/4FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP«ó³fNïbI &Purity/*FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPƾƒ/JðbE "Purity/*FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP—RÌÔNñbI &Purity/*FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPK‹|ÿNòbI &Purity/0FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPQ—á­NóbI &Purity/0FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP/ŽbNôbI &Purity/1FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP&J¥NõbI &Purity/2FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPt¯1]JöbE "Purity/2FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP1W7çJ÷bE "Purity/*FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP”øKJøbE "Purity/1FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP*ô¹åJùbE "Purity/1FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP¿^CNúbI &Purity/2FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP7CYINûbI &Purity/4FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPáD"HübC Purity/0FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JP¹FJýbE "Purity/0FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP,ló;þb6 jet/accuracy_7_of_7"trainer/global_step2:JP9‚¢HÿbC Purity/4FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP,¾ùN€bI &Purity/*FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP—OpMNbI &Purity/1FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP„IóJ‚bE "Purity/1FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP &{îHƒbC Purity/3FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPF‰üJ„bE "Purity/*FRt4SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPˆQ"@…b; particle/accuracy_3_of_6"trainer/global_step2:JP¢e[ H†bC Purity/0FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPÓèAÈH‡bC Purity/1FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JP‚ªjHˆbC Purity/*FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPqcÝN‰bI &Purity/*FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP~Š‹ÒHŠbC Purity/1FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP¨c˜ý@‹b; particle/accuracy_1_of_2"trainer/global_step2:JPÓ±'JŒbE "Purity/*FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPrŽ[hJbE "Purity/*FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP±,J-HŽbC Purity/1FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPn¢†JbE "Purity/1FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP~ÖxNbI &Purity/2FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP nÐH‘bC Purity/2FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPE Á†H’bC Purity/2FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP¯a|N“bI &Purity/2FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP ›­³J”bE "Purity/*FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP½†àJ•bE "Purity/1FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPd¼7NN–bI &Purity/2FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPùòÙN—bI &Purity/2FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP©»äÚH˜bC Purity/3FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JPp—EI@™b; particle/accuracy_5_of_5"trainer/global_step2:JPI¹6ÂJšbE "Purity/3FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPß`Ý@›b; particle/accuracy_3_of_7"trainer/global_step2:JPOæX‡JœbE "Purity/*FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPOœç×NbI &Purity/0FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPľ7NžbI &Purity/3FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP*bØ;Ÿb6 jet/accuracy_6_of_6"trainer/global_step2:JPu2·æH bC Purity/0FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPæíCöH¡bC Purity/2FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPœR)J¢bE "Purity/2FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPxAºLN£bI &Purity/3FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP×m&N¤bI &Purity/4FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP›õA H¥bC Purity/*FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPZ9d„H¦bC Purity/*FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPÙéŸãJ§bE "Purity/0FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP… 9N¨bI &Purity/1FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPY^ÙJ©bE "Purity/2FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP@Ì JªbE "Purity/3FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPìªPH«bC Purity/0FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPA£ùH¬bC Purity/2FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPžŒrH­bC Purity/2FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JP@‰qJ®bE "Purity/*FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP$Ôµ(N¯bI &Purity/*FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP‚\H¤N°bI &Purity/0FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP¼âÉBJ±bE "Purity/2FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPVøN²bI &Purity/3FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP ]¨7J³bE "Purity/3FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP{×\¥J´bE "Purity/*FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP~ÎõHµbC Purity/*FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JP } šH¶bC Purity/0FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPá(”øJ·bE "Purity/1FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPJ´ßH¸bC Purity/3FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP›TŒnJ¹bE "Purity/1FRt*SRqqt4FBt/event_purity"trainer/global_step2:JP&RpÝJºbE "Purity/0FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPvU2'H»bC Purity/*FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP@¿zH¼bC Purity/*FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPJˆ§J½bE "Purity/*FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP¼5ºN¾bI &Purity/2FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPÔqÈJ¿bE "Purity/3FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPÕf°ÍJÀbE "Purity/0FRt1SRqqt3FBt/event_purity"trainer/global_step2:JPNMÖJÁbE "Purity/4FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPx2M¡HÂbC Purity/4FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPHº01NÃbI &Purity/*FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPn’eèJÄbE "Purity/0FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP‘ט›JÅbE "Purity/0FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPŽ¥ 2NÆbI &Purity/0FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP]\FÈJÇbE "Purity/1FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPu„ÿNÈbI &Purity/1FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPr†­IHÉbC Purity/2FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP7“@NÊbI &Purity/1FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP84­ªHËbC Purity/1FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPzŒ„NÌbI &Purity/1FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP¾“ÌÓNÍbI &Purity/1FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP¯ó¯±HÎbC Purity/1FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPJÏbE "Purity/2FRt3SRqqt1FBt/event_purity"trainer/global_step2:JP—*jHÐbC Purity/2FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP{+¦þJÑbE "Purity/3FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPV¼ÍžNÒbI &Purity/0FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPï̲HÓbC Purity/1FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPµNIJÔbE "Purity/3FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP‘Ø +HÕbC Purity/3FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPÁ f_NÖbI &Purity/3FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPÊEGH×bC Purity/*FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPÑ=•šHØbC Purity/*FRt1SRqqt4FBt/FRt_purity"trainer/global_step2:JPcDF~JÙbE "Purity/4FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPÐêtJÚbE "Purity/0FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPÑ5%HÛbC Purity/1FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP²ÃHÜbC Purity/2FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP²TnHÝbC Purity/3FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JP$ˆÊJÞbE "Purity/3FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPÉœpJßbE "Purity/0FRt4SRqqt1FBt/event_purity"trainer/global_step2:JPŠðÝJàbE "Purity/0FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPgùÖ§JábE "Purity/0FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPå#=]JâbE "Purity/1FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPqšÏ?JãbE "Purity/3FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPöÚ'¸NäbI &Purity/3FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPt…Ÿ¯@åb; particle/accuracy_3_of_3"trainer/global_step2:JP9í/ÝJæbE "Purity/*FRt1SRqqt3FBt/event_purity"trainer/global_step2:JP2] NçbI &Purity/*FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPÿ0“GHèbC Purity/1FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JP¾sHébC Purity/3FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPüöÖ;êb6 jet/accuracy_2_of_6"trainer/global_step2:JPô±m³HëbC Purity/0FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP£‚\NìbI &Purity/1FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPâK—NíbI &Purity/2FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP—…t$JîbE "Purity/3FRt*SRqqt1FBt/event_purity"trainer/global_step2:JPhºpNïbI &Purity/3FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPV‡rJðbE "Purity/0FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPÅò?ªHñbC Purity/*FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP`0ŒOHòbC Purity/2FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP5ZœHóbC Purity/1FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JPÈlNôbI &Purity/*FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP£‹Ç‘JõbE "Purity/2FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÓŽ¾JöbE "Purity/3FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPz¹_N÷bI &Purity/4FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP½ŸVNøbI &Purity/3FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP_ D&NùbI &Purity/0FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP?’µhNúbI &Purity/2FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPê}Q%JûbE "Purity/2FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPóÖ!NübI &Purity/2FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPö½o)HýbC Purity/3FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPM¸/ÎHþbC Purity/*FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JP)¥NÿbI &Purity/*FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPß}/H€bC Purity/*FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPqøùÉJbE "Purity/0FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPÏXH‚bC Purity/2FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP~.œNƒbI &Purity/2FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP¦9"¼H„bC Purity/2FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JP'VÊ;…b6 jet/accuracy_3_of_3"trainer/global_step2:JPîÝU@†b; particle/accuracy_4_of_4"trainer/global_step2:JPDÄRH‡bC Purity/4FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP9$¸JˆbE "Purity/*FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP•(q“N‰bI &Purity/*FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP“§µ¤NŠbI &Purity/0FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPeèN‹bI &Purity/2FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP¶§ÐJŒbE "Purity/3FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP{ bäJbE "Purity/0FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPôê©VNŽbI &Purity/0FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPy+ã‘NbI &Purity/1FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP ¼ñJbE "Purity/1FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP³ðʤJ‘bE "Purity/2FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPh?aÅJ’bE "Purity/2FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP_ˆZAN“bI &Purity/3FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPj ê“N”bI &Purity/3FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPŸ?¾3J•bE "Purity/0FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPôlÛÄJ–bE "Purity/0FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÞ¥5N—bI &Purity/1FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP€Ù£H˜bC Purity/1FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPl[Å¥H™bC Purity/1FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP-Ë¿þHšbC Purity/*FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPÖŽHJ›bE "Purity/0FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPTg_LHœbC Purity/3FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP,ƘNbI &Purity/2FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP«o­@žb; particle/accuracy_1_of_3"trainer/global_step2:JP_ÛïJŸbE "Purity/0FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPöõœkH bC Purity/4FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPí‡ì«H¡bC Purity/*FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPþœ‡H¢bC Purity/2FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPÓž¿ÞN£bI &Purity/2FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPI}I8¤b3 particle/f_score"trainer/global_step2:JPJ.98H¥bC Purity/*FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JP*Ãa:J¦bE "Purity/4FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPãÑsN§bI &Purity/*FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP¾ïH¨bC Purity/*FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPt¨CH©bC Purity/0FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPøͶHªbC Purity/1FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPm!›æN«bI &Purity/3FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPfìýN¬bI &Purity/4FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPÞ¿ÐHN­bI &Purity/0FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPRÓ•N®bI &Purity/1FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP¨þž3H¯bC Purity/3FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPCwÅÎJ°bE "Purity/4FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPq4Ó¨J±bE "Purity/*FRt2SRqqt3FBt/event_purity"trainer/global_step2:JPµÈš(J²bE "Purity/*FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPïºiËN³bI &Purity/*FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP›MëJ´bE "Purity/1FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPÍ–»HµbC Purity/2FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JPô8 cN¶bI &Purity/*FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPÓ±‰H·bC Purity/*FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPìT¬OH¸bC Purity/1FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPðå,N¹bI &Purity/3FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPöÎ)®NºbI &Purity/4FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPHŽdH»bC Purity/*FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPðöyH¼bC Purity/2FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP~¶õN½bI &Purity/3FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPÇšµL@¾b; particle/accuracy_2_of_2"trainer/global_step2:JP¦e8÷J¿bE "Purity/0FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP™ÈìöJÀbE "Purity/4FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP›[¾NÁbI &Purity/4FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPó JJÂbE "Purity/2FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPtû¥NÃbI &Purity/*FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPGO™8JÄbE "Purity/0FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP)?S,JÅbE "Purity/1FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP28íNÆbI &Purity/1FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPºÄr-NÇbI &Purity/1FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPµ"9JÈbE "Purity/*FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPcZ’HÉbC Purity/3FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JP˜¤úNÊbI &Purity/0FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPÔ˜îHËbC Purity/1FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPÒäšHÌbC Purity/1FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP1åWÛHÍbC Purity/2FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPƒ›¬JÎbE "Purity/2FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPw­v=9Ïb4 particle/accuracy"trainer/global_step2:JP™‰JÐbE "Purity/*FRt4SRqqt1FBt/event_purity"trainer/global_step2:JP¹aÞ`;Ñb6 jet/accuracy_1_of_6"trainer/global_step2:JP³4‰HÒbC Purity/*FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP„H…aJÓbE "Purity/*FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÕºvHÔbC Purity/1FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP©û@Õb; particle/accuracy_3_of_4"trainer/global_step2:JPÈJ©ŸHÖbC Purity/2FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JPEiñtJ×bE "Purity/3FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPY}þ;Øb6 jet/accuracy_6_of_7"trainer/global_step2:JPöâ%—HÙbC Purity/*FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JPÅþc½NÚbI &Purity/*FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP56ñˆJÛbE "Purity/1FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP·qÅÂJÜbE "Purity/1FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPh0 ýJÝbE "Purity/2FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPO6A;Þb6 jet/accuracy_2_of_3"trainer/global_step2:JP3­ü%HßbC Purity/3FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPôôœHàbC Purity/*FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JPu ¾NábI &Purity/*FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP.nûÝNâbI &Purity/0FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPcµËÎNãbI &Purity/0FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPãöNäbI &Purity/1FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPó7¹JåbE "Purity/1FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPV=Á%NæbI &Purity/3FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP— ä @çb; particle/accuracy_2_of_4"trainer/global_step2:JP¤˜ÕÉ<èb7 particle/sensitivity"trainer/global_step2:JP«®1HébC Purity/*FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JP™‹S^JêbE "Purity/2FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP‘iJëbE "Purity/0FRt*SRqqt4FBt/event_purity"trainer/global_step2:JPÐ…„ÜJìbE "Purity/1FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPÐÐŒHíbC Purity/2FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JP¾ÈÙ+JîbE "Purity/*FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP¦­ úHïbC Purity/*FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPUQ€âHðbC Purity/0FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPFPl|NñbI &Purity/0FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP©GNøNòbI &Purity/2FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP_”%¨JóbE "Purity/3FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPê`ÄŠ;ôb6 jet/accuracy_2_of_5"trainer/global_step2:JPˆcÔ JõbE "Purity/2FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPp¤ JöbE "Purity/1FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP¯˜ÍÐJ÷bE "Purity/2FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPw¿QJøbE "Purity/3FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP»ø’Š@ùb; particle/accuracy_4_of_5"trainer/global_step2:JPA¬Ïá;úb6 jet/accuracy_4_of_7"trainer/global_step2:JP¯3YCJûbE "Purity/1FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP7qlyHübC Purity/1FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPÌóN6JýbE "Purity/*FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP–[VýNþbI &Purity/4FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP,´†€@ÿb; particle/accuracy_3_of_5"trainer/global_step2:JPOø4}Ìšß?€b; particle/accuracy_1_of_6"trainer/global_step2:JP!‡»ŒHbC Purity/1FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JP¦˜?J‚bE "Purity/1FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP ©JƒbE "Purity/2FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPF¯wH„bC Purity/2FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPãë]Ö;…b6 jet/accuracy_3_of_7"trainer/global_step2:JP+W·J†bE "Purity/3FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP‹p¦ÔJ‡bE "Purity/*FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP“¥JJˆbE "Purity/*FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPoö*J½bE "Purity/0FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP>VjÊJ¾bE "Purity/1FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPI>J¿bE "Purity/1FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPÉ8GNÀbI &Purity/4FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP‚X}ûNÁbI &Purity/4FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÄM?4HÂbC Purity/*FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPæóáíHÃbC Purity/3FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JP^o÷æ;Äb6 jet/accuracy_5_of_7"trainer/global_step2:JPxr¡IJÅbE "Purity/2FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPÝ*èfJÆbE "Purity/3FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPW 3XJÇbE "Purity/3FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP{åÞtHÈbC Purity/3FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPýê7HÉbC Purity/*FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JP_ôýéJÊbE "Purity/3FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP:0yÅNËbI &Purity/3FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPT¦…+NÌbI &Purity/3FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPÆy"NÍbI &Purity/4FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPÐ: NÎbI &Purity/*FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPµG2uJÏbE "Purity/1FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPŸü§UNÐbI &Purity/1FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP‚ĘHÑbC Purity/2FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP(å4âJÒbE "Purity/0FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP&-ŽNÓbI &Purity/*FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP³núXNÔbI &Purity/0FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPAg[JÕbE "Purity/*FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP£‘¶JÖbE "Purity/*FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP §µ¾N×bI &Purity/0FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP½пHØbC Purity/2FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPPåa¼JÙbE "Purity/2FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP_oq"JÚbE "Purity/1FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JPÀ}‰JÛbE "Purity/*FRt3SRqqt2FBt/event_purity"trainer/global_step2:JPÆxò`NÜbI &Purity/*FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPI ³SHÝbC Purity/1FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP~|6HJÞbE "Purity/2FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP€ï–½JßbE "Purity/*FRt*SRqqt4FBt/event_purity"trainer/global_step2:JPÊUÕbJàbE "Purity/1FRt3SRqqt2FBt/event_purity"trainer/global_step2:JPûÉJábE "Purity/0FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÎF7NâbI &Purity/1FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPÉM…kNãbI &Purity/2FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPn&Z*NäbI &Purity/4FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPÞH¬JåbE "Purity/*FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPWQ/õ@æb; particle/accuracy_6_of_6"trainer/global_step2:JPÆðCHçbC Purity/3FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPbý[JèbE "Purity/*FRt1SRqqt4FBt/event_purity"trainer/global_step2:JPØaÎÇHébC Purity/*FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPÿaÃVNêbI &Purity/0FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPý ¸ NëbI &Purity/4FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPªRÚ¤NìbI &Purity/4FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPNõnHHíbC Purity/*FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP 7ù;îb6 jet/accuracy_2_of_4"trainer/global_step2:JPÙ(NXHïbC Purity/*FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPyS‹.NðbI &Purity/*FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPà HñbC Purity/*FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JP vå`HòbC Purity/1FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPç<[ÈNóbI &Purity/3FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPÓÁ»ýNôbI &Purity/4FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPÉ'9NõbI &Purity/0FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP䦾žJöbE "Purity/2FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPýJñH÷bC Purity/2FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPñßkdGøbB validation_average_jet_accuracy"trainer/global_step2:JPcÛisHùbC Purity/0FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JPQä\JúbE "Purity/4FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP=;IJûbE "Purity/*FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP8„ÚDNübI &Purity/0FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPÖ+ÄNýbI &Purity/3FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP%Œm@þb; particle/accuracy_1_of_4"trainer/global_step2:JPkË6NÿbI &Purity/*FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPœŒUvN€ bI &Purity/1FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP<Ò=N bI &Purity/3FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JP«ÌJ‚ bE "Purity/2FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPƒèn=Hƒ bC Purity/*FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP*sIPJ„ bE "Purity/1FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP¦–¢tJ… bE "Purity/2FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPØ  N† bI &Purity/3FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPï"ËmJ‡ bE "Purity/3FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPeÒ+Jˆ bE "Purity/4FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPZX¤xN‰ bI &Purity/0FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP#yz";Š b6 jet/accuracy_2_of_2"trainer/global_step2:JPg-7”@‹ b; particle/accuracy_1_of_1"trainer/global_step2:JPÿ »òHŒ bC Purity/0FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPêÞÑ; b6 jet/accuracy_3_of_6"trainer/global_step2:JPÛ`"JŽ bE "Purity/3FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPrV-N bI &Purity/1FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP“–†]@ b; particle/accuracy_1_of_7"trainer/global_step2:JP-ªzH‘ bC Purity/1FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JPâ^RH’ bC Purity/*FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPgˆaH“ bC Purity/1FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP.}y"N” bI &Purity/1FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPZÖ*H• bC Purity/2FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPFh`ùN– bI &Purity/2FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP: /½N— bI &Purity/2FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPÈz-J˜ bE "Purity/3FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP²òaËJ™ bE "Purity/*FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPãêvHš bC Purity/*FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JP7ûT\H› bC Purity/1FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JP:X!³Hœ bC Purity/2FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP<5œíJ bE "Purity/3FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPø•"}Jž bE "Purity/*FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPW'ĵNŸ bI &Purity/1FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPTÎ%hN  bI &Purity/4FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP”„Ì&N¡ bI &Purity/4FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPªÞ¾ØJ¢ bE "Purity/3FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPž# J£ bE "Purity/1FRt4SRqqt*FBt/event_purity"trainer/global_step2:JP¥8Á5J¤ bE "Purity/0FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPÀ½Ï"J¥ bE "Purity/0FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPì@ãJ¦ bE "Purity/1FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPýã†:J§ bE "Purity/1FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPwç©7H¨ bC Purity/1FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JP“²ªJ© bE "Purity/3FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPOÁÕHª bC Purity/3FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPÎ.œ@« b; particle/accuracy_5_of_6"trainer/global_step2:JPÒêÁ6N¬ bI &Purity/0FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP¸7=kJ­ bE "Purity/2FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÓ3±öN® bI &Purity/4FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP€¸A‰@¯ b; particle/accuracy_1_of_5"trainer/global_step2:JPd”lJ° bE "Purity/*FRt0SRqqt4FBt/event_purity"trainer/global_step2:JP4HŽŠH± bC Purity/1FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JP—YH² bC Purity/3FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JP3’àJ³ bE "Purity/4FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP® N´ bI &Purity/0FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JPÿ‹¹Nµ bI &Purity/2FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JP(‚²ÿN¶ bI &Purity/3FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPÜîsN· bI &Purity/4FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPXÎvN¸ bI &Purity/*FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPíŇJ¹ bE "Purity/*FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPt‘²yNº bI &Purity/2FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPP’jµH» bC Purity/*FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPééþ&H¼ bC Purity/1FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JP¼AfH½ bC Purity/2FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPàs ÌH¾ bC Purity/4FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPR vN¿ bI &Purity/2FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP!5 _JÀ bE "Purity/2FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPþ}½ïNÁ bI &Purity/2FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPÇâéþJ bE "Purity/3FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÄX«èHà bC Purity/2FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JP§IÐ@Ä b; particle/accuracy_6_of_7"trainer/global_step2:JPޱߌJÅ bE "Purity/*FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÓœÝ]JÆ bE "Purity/1FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP —$4JÇ bE "Purity/1FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPJ­›NÈ bI &Purity/2FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPÕï¬XNÉ bI &Purity/2FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP ²’hJÊ bE "Purity/3FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPèïNË bI &Purity/4FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPV¹)HÌ bC Purity/*FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JP¬‡'jHÍ bC Purity/*FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPÄíüDHÎ bC Purity/0FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPž^ÐHÏ bC Purity/0FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPû­†»JÐ bE "Purity/1FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPkÄ} NÑ bI &Purity/1FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP–£HÒ bC Purity/3FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP“ ÒNÓ bI &Purity/4FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPÝŸS³;Ô b6 jet/accuracy_1_of_1"trainer/global_step2:JPi2¡’JÕ bE "Purity/*FRt1SRqqt2FBt/event_purity"trainer/global_step2:JP6³´HÖ bC Purity/1FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JP¶ùŸºN× bI &Purity/2FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP… ߥNØ bI &Purity/3FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP§”ÊÒ0Ù + _wandbruntime‚557  _runtime‚557ôi×7;Ú ·¥ ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_4‚0.02375815436244011 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.055018387734889984 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.23135706782341003 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.11008232086896896 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.16875869035720825 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3121769428253174 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8259261250495911 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 9 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.030193235725164413 *jet/accuracy_1_of_3‚0.7254094481468201 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.02436419017612934 jet/accuracy_5_of_5‚0 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5333333611488342 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.010077361017465591 8"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.379616379737854 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26886308193206787 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 -jet/accuracy_4_of_5‚0.0067588333040475845 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2152777910232544 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 particle/accuracy_7_of_7‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 particle/accuracy_4_of_7‚1 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0017785715172067285 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.23629508912563324 8"Purity/*FRt0SRqqt2FBt/event_purity‚0.264976441860199 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.023941392078995705 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_2_of_6‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2894737124443054 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.046417348086833954 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.10353536158800125 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.40383902192115784 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.33366692066192627 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.180304154753685 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.33253970742225647 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 jet/accuracy_7_of_7‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.2850787937641144 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0462169386446476 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_3_of_6‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9931385517120361 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.05581023171544075 particle/accuracy_1_of_2‚1 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6213971376419067 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.008156738243997097 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.10130881518125534 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.009509710595011711 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.055552154779434204 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.08409523963928223 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3482030928134918 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.20512822270393372 particle/accuracy_5_of_5‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_7‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2682593762874603 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 jet/accuracy_6_of_6‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9873272180557251 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.245555579662323 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.005058007780462503 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 9 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.054190754890441895 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9956194162368774 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3273523449897766 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4061444103717804 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666150093079 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.06946422159671783 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809850215912 _runtime‚ 557.265861242 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.35186028480529785 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9902364015579224 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9922847747802734 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.34540820121765137 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.19871793687343597 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.10810354351997375 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.06695780903100967 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.29141610860824585 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.14814813435077667 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.3798533082008362 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9935715198516846 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9901549816131592 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.25225529074668884 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.03587963059544563 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.10989131033420563 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2773425579071045 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.04444444552063942 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3393591344356537 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3481481373310089 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.15429294109344482 +jet/accuracy_2_of_6‚0.21052631735801697 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9948790669441223 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.24344764649868011 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.11817886680364609 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.289498895406723 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.34567904472351074 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5394737124443054 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.03267974033951759 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8968254327774048 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.17222221195697784 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.01885942742228508 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 _step‚48 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.16458334028720856 ,jet/accuracy_3_of_3‚0.010584385134279728 particle/accuracy_4_of_4‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2976190149784088 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3059758245944977 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.2730288803577423 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.2999950349330902 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.36428573727607727 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3793410062789917 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.3465803563594818 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9956097602844238 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9897156953811646 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.27412280440330505 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 %particle/f_score‚0.28532674908638 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.07837484776973724 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.3749169409275055 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.358443945646286 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.21931874752044678 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.07696982473134995 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9996017217636108 9 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.056844666600227356 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 particle/accuracy_2_of_2‚1 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.37388885021209717 ;"Purity/1FRt*SRqqt1FBt/event_puri™{í‡Wty‚0.047977570444345474 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9863181114196777 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9875576496124268 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2589722275733948 (particle/accuracy‚0.8217406272888184 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9736841917037964 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.12566444277763367 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2875767946243286 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 particle/accuracy_3_of_4‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 jet/accuracy_6_of_7‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.06553670018911362 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.39993855357170105 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2771487832069397 *jet/accuracy_2_of_3‚0.1554846614599228 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 +particle/sensitivity‚0.2279476672410965 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9662308096885681 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3358008861541748 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9919071793556213 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993913769721985 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3385416567325592 +jet/accuracy_2_of_5‚0.26719292998313904 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 <"Purity/1FRt1SRqqt*FBt/event_purity‚0.0035275383852422237 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.28035715222358704 particle/accuracy_4_of_5‚1 jet/accuracy_4_of_7‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3214285671710968 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3241145610809326 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 particle/accuracy_1_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 jet/accuracy_3_of_7‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809850215912 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.2757604420185089 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9880644083023071 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9624074101448059 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.28865519165992737 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.29934564232826233 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.08579947054386139 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.30055859684944153 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,jet/accuracy_3_of_5‚0.030924810096621513 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.24951913952827454 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 9 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.054006658494472504 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2790706753730774 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.27837619185447693 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_4_of_6‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9907862544059753 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7"Purity/0FRt*SRqqt1FBt/event_purity‚0.61174076795578 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.08904723078012466 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 & Purity/*FRt1SRqqt4FBt/FBt_purity‚1 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5333333611488342 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32611650228500366 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6904762387275696 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.10754088312387466 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.05624425411224365 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.008994259871542454 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3273523449897766 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_6_of_6‚1 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.29411765933036804 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.05809120088815689 *jet/accuracy_2_of_4‚0.2377772331237793 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9856930375099182 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.24770405888557434 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2971775233745575 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 6validation_average_jet_accuracy‚0.3407520651817322 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10344826430082321 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_1_of_4‚1 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9839576482772827 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08357144147157669 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.020370371639728546 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3375000059604645 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 +jet/accuracy_2_of_2‚0.06412088125944138 particle/accuracy_1_of_1‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_3_of_6‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 % _timestamp‚1.7883898015079074e+09 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.05463343486189842 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9943825602531433 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1930428147315979 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.29074665904045105 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9687499403953552 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.13551929593086243 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993913769721985 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.33330288529396057 8"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.318337082862854 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.13333334028720856 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_5_of_6‚1 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5745614171028137 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 trainer/global_step‚603 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9246031641960144 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.17910906672477722 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.26547616720199585 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.32608696818351746 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 particle/accuracy_6_of_7‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3147650361061096 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3271889090538025 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29554107785224915 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.9166666865348816 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.16145090758800507 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9851303696632385 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.980262815952301 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05594037473201752 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.3787630498409271 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.009384163655340672 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.34313878417015076 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8269714117050171 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.2720426917076111 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.2730158269405365 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.25736552476882935 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.3125 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 *jet/accuracy_1_of_4‚0.7758224010467529 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ;"Purity/*FRt1SRqqt1FBt/event_purity‚0.047820594161748886 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.31483641266822815 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.932222306728363 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.05581023171544075 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_4‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 epoch‚0 )loss/val_total_loss‚56.65863875356903 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.05841014161705971 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.1687936633825302 particle/accuracy_2_of_3‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.24940131604671478 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.01833333447575569 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3265105187892914 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9852564930915833 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.30000001192092896 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.10368990153074265 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9207819700241089 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.23362435400485992 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 *jet/accuracy_1_of_2‚0.6144313812255859 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2421875 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1944444626569748 jet/accuracy_2_of_7‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.3918665051460266 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3388310968875885 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.08618367463350296 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.26069024205207825 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.031211569905281067 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.30054736137390137 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.23297971487045288 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.05355027690529823 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.10909205675125122 9 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.060281384736299515 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9919636249542236 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.12928448617458344 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 +particle/specificity‚0.9316493272781372 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.21794873476028442 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.18666666746139526 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.35479938983917236 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2265625 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1309523731470108 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1983766108751297 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7380952835083008 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375000596046448 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0019315123790875077 particle/accuracy_2_of_5‚1 particle/accuracy_5_of_7‚1 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.08728599548339844 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_4_of_6‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.055415790528059006 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9921844005584717 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.07604334503412247 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.32548707723617554 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9920907616615295 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.10415658354759216 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.05499923229217529 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9224137663841248 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3879440426826477 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9900477528572083 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.37375217676162724Luk(Û »¥ ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.009384163655340672 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.34313878417015076 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8269714117050171 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.2720426917076111 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.2730158269405365 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.25736552476882935 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.3125 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 *jet/accuracy_1_of_4‚0.7758224010467529 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ;"Purity/*FRt1SRqqt1FBt/event_purity‚0.047820594161748886 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.31483641266822815 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.932222306728363 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.05581023171544075 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_4‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 epoch‚0 )loss/val_total_loss‚56.65863875356903 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.05841014161705971 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.1687936633825302 particle/accuracy_2_of_3‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.24940131604671478 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.01833333447575569 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3265105187892914 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9852564930915833 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.30000001192092896 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.10368990153074265 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9207819700241089 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.23362435400485992 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 *jet/accuracy_1_of_2‚0.6144313812255859 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2421875 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1944444626569748 jet/accuracy_2_of_7‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.3918665051460266 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3388310968875885 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.08618367463350296 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.26069024205207825 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.031211569905281067 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.30054736137390137 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.23297971487045288 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.05355027690529823 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.10909205675125122 9 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.060281384736299515 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9919636249542236 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.12928448617458344 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 +particle/specificity‚0.9316493272781372 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.21794873476028442 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.18666666746139526 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.35479938983917236 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2265625 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1309523731470108 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1983766108751297 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7380952835083008 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375000596046448 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0019315123790875077 particle/accuracy_2_of_5‚1 particle/accuracy_5_of_7‚1 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.08728599548339844 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_4_of_6‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.055415790528059006 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9921844005584717 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.07604334503412247 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.32548707723617554 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9920907616615295 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.10415658354759216 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.05499923229217529 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9224137663841248 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3879440426826477 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9900477528572083 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.3737521767616272 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_4‚0.02375815436244011 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.055018387734889984 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.23135706782341003 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.11008232086896896 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.16875869035720825 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3121769428253174 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8259261250495911 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 9 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.030193235725164413 *jet/accuracy_1_of_3‚0.7254094481468201 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.02436419017612934 jet/accuracy_5_of_5‚0 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5333333611488342 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.010077361017465591 8"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.379616379737854 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt2SRqqt1FBt/evBÀ¶ßWjent_proportion‚0.02255544438958168 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26886308193206787 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 -jet/accuracy_4_of_5‚0.0067588333040475845 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2152777910232544 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 particle/accuracy_7_of_7‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 particle/accuracy_4_of_7‚1 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0017785715172067285 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.23629508912563324 8"Purity/*FRt0SRqqt2FBt/event_purity‚0.264976441860199 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.023941392078995705 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_2_of_6‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2894737124443054 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.046417348086833954 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.10353536158800125 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.40383902192115784 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.33366692066192627 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.180304154753685 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.33253970742225647 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 jet/accuracy_7_of_7‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.2850787937641144 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0462169386446476 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_3_of_6‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9931385517120361 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.05581023171544075 particle/accuracy_1_of_2‚1 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6213971376419067 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.008156738243997097 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.10130881518125534 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.009509710595011711 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.055552154779434204 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.08409523963928223 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3482030928134918 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.20512822270393372 particle/accuracy_5_of_5‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_7‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2682593762874603 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 jet/accuracy_6_of_6‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9873272180557251 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.245555579662323 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.005058007780462503 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 9 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.054190754890441895 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9956194162368774 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3273523449897766 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4061444103717804 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666150093079 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.06946422159671783 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809850215912 _runtime‚ 557.265861242 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.35186028480529785 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9902364015579224 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9922847747802734 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.34540820121765137 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.19871793687343597 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.10810354351997375 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.06695780903100967 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.29141610860824585 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.14814813435077667 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.3798533082008362 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9935715198516846 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9901549816131592 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.25225529074668884 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.03587963059544563 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.10989131033420563 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2773425579071045 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.04444444552063942 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3393591344356537 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3481481373310089 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.15429294109344482 +jet/accuracy_2_of_6‚0.21052631735801697 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9948790669441223 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.24344764649868011 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.11817886680364609 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.289498895406723 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.34567904472351074 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5394737124443054 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.03267974033951759 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8968254327774048 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.17222221195697784 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.01885942742228508 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 _step‚48 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.16458334028720856 ,jet/accuracy_3_of_3‚0.010584385134279728 particle/accuracy_4_of_4‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2976190149784088 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3059758245944977 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.2730288803577423 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.2999950349330902 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.36428573727607727 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3793410062789917 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.3465803563594818 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9956097602844238 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9897156953811646 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.27412280440330505 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 %particle/f_score‚0.28532674908638 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.07837484776973724 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.3749169409275055 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.358443945646286 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.21931874752044678 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.07696982473134995 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9996017217636108 9 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.056844666600227356 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 particle/accuracy_2_of_2‚1 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.37388885021209717 ;"Purity/1FRt*SRqqt1FBt/event_purity‚0.047977570444345474 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9863181114196777 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9875576496124268 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2589722275733948 (particle/accuracy‚0.8217406272888184 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9736841917037964 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.12566444277763367 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2875767946243286 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 particle/accuracy_3_of_4‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 jet/accuracy_6_of_7‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.06553670018911362 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.39993855357170105 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2771487832069397 *jet/accuracy_2_of_3‚0.1554846614599228 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 +particle/sensitivity‚0.2279476672410965 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9662308096885681 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3358008861541748 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9919071793556213 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993913769721985 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3385416567325592 +jet/accuracy_2_of_5‚0.26719292998313904 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 <"Purity/1FRt1SRqqt*FBt/event_purity‚0.0035275383852422237 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.28035715222358704 particle/accuracy_4_of_5‚1 jet/accuracy_4_of_7‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3214285671710968 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3241145610809326 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 particle/accuracy_1_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 jet/accuracy_3_of_7‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809850215912 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.2757604420185089 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9880644083023071 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9624074101448059 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.28865519165992737 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.29934564232826233 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.08579947054386139 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.30055859684944153 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,jet/accuracy_3_of_5‚0.030924810096621513 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.24951913952827454 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 9 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.054006658494472504 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2790706753730774 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.27837619185447693 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_4_of_6‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9907862544059753 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7"Purity/0FRt*SRqqt1FBt/event_purity‚0.61174076795578 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.08904723078012466 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 & Purity/*FRt1SRqqt4FBt/FBt_purity‚1 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5333333611488342 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32611650228500366 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6904762387275696 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.10754088312387466 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.05624425411224365 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.008994259871542454 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3273523449897766 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_6_of_6‚1 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.29411765933036804 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.05809120088815689 *jet/accuracy_2_of_4‚0.2377772331237793 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9856930375099182 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.24770405888557434 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2971775233745575 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 6validation_average_jet_accuracy‚0.3407520651817322 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10344826430082321 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_1_of_4‚1 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9839576482772827 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08357144147157669 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.020370371639728546 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3375000059604645 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 +jet/accuracy_2_of_2‚0.06412088125944138 particle/accuracy_1_of_1‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_3_of_6‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 % _timestamp‚1.7883898015079074e+09 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.05463343486189842 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9943825602531433 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1930428147315979 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.29074665904045105 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9687499403953552 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.13551929593086243 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993913769721985 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.33330288529396057 8"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.318337082862854 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.13333334028720856 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_5_of_6‚1 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5745614171028137 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 trainer/global_step‚603 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9246031641960144 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.17910906672477722 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.26547616720199585 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.32608696818351746 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 particle/accuracy_6_of_7‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3147650361061096 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3271889090538025 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29554107785224915 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.9166666865348816 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.16145090758800507 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9851303696632385 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.980262815952301 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05594037473201752 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.37876304984092710“%Ü0Ü ‚(:’ ¶ÓâÔæã©u gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.256' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.051199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‚.€&Ý ‚(:› ¶ÓâÔÚÏþw% disk./.usageGB‚1042.6200866699219 disk.rbd3.in‚0 disk.sda3.in‚0$ memory_percent‚83.35154056549072 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚ 39.7109375' disk.nvme0c0n1.out‚2962.32861328125 proc.cpu.threads‚17 network.recv‚255913) disk./.usagePercent‚69.97110883293989 disk.drbd1777.out‚0 cpu‚23.04963685269543 disk.dm-1.out‚ 23.9453125 disk.rbd2.in‚0 disk.rbd2.out‚ 22.9990234375% disk.nvme0n1.out‚2962.32861328125 disk.rbd0.in‚0 disk.dm-0.in‚ 39.7109375) proc.memory.availableMB‚ 2727.68359375 disk.sda.in‚0 disk.rbd0.out‚0.078125 disk.rbd3.out‚ 1.28515625# proc.memory.rssMB‚ 2902.39453125* proc.memory.percent‚17.714810371398926 disk.sda.out‚ 23.9453125 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.nvme0n1.in‚ 39.7109375 disk.sda1.out‚0 disk.sda2.out‚0 disk.sda3.out‚ 23.9453125 disk.rbd1.out‚ 0.306640625 disk.sda1.in‚0" disk.dm-0.out‚2962.32861328125 network.sent‚843570 disk.sda2.in‚0Ö¾{Þ ‚(:’ ÅÓâÔÿ«¾u gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.909' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.181799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0í/N&ß ‚(:› ÅÓâÔ‡€x network.recv‚261878 disk.rbd2.in‚0 disk.sda.out‚ 24.5390625' disk.nvme0c0n1.out‚2962.61376953125 disk.dm-0.in‚ 39.7109375" disk.dm-0.out‚2962.61376953125 disk.sda3.out‚ 24.5390625$ memory_percent‚83.35137367248535 disk.sda.in‚0 disk.sda1.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.306640625% disk.nvme0n1.out‚2962.61376953125 cpu‚23.106216369610863) disk./.usagePercent‚69.97110883293989% disk./.usageGB‚1042.6200866699219 disk.nvme0n1.in‚ 39.7109375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 proc.cpu.threads‚17 network.sent‚946101 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 39.7109375 disk.sda3.in‚0( proc.memory.availableMB‚ 2727.7109375 disk.rbd2.out‚ 24.0732421875 disk.sda2.out‚0 disk.dm-1.out‚ 24.5390625 disk.drbd1777.out‚0 disk.rbd3.out‚ 1.28515625# proc.memory.rssMB‚ 2902.39453125 disk.dm-1.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0* proc.memory.percent‚17.714810371398926°AtZ0à + _wandbruntime‚597  _runtime‚597XDJÄá ¾ _step‚49 & lr-AdamW/pg1‚0.0017183009882271786 & lr-AdamW/pg2‚0.0017183009882271786 trainer/global_step‚649 % _timestamp‚1.7883898421706152e+09 _runtime‚ 597.926603354˘‹Èâ  % _timestamp‚1.7883898421706152e+09 _runtime‚ 597.926603354 _step‚49 & lr-AdamW/pg1‚0.0017183009882271786 & lr-AdamW/pg2‚0.0017183009882271786 trainer/global_step‚6491\ÿá0ã + _wandbruntime‚597  _runtime‚597ÞÈ°ñä ¾O dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚44084166809088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3479054 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11141966 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _runtime‚ 597.92803411 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 535677085696 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚482 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42670365435904 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 532495206239 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43732112626688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13849180 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚442675935344644¥%ÕÓ ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 532687202771 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚50 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231641088 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 555634618880 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13871776 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42670348396544 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚44267806751744 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚44640033069056 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚44084141905408 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚44267593534464 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 535480907776 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 trainer/global_step‚649 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 555866259968 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24991146 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10306766 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚44639776524288iiNµÈ'å ÂO XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚44267806751744 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚44640033069056 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚44084141905408 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚44267593534464 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 535480907776 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 trainer/global_step‚649 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 555866259968 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24991146 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10306766 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚44639776524288 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚44084166809088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3479054 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11141966 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _runtime‚ 597.92803411 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 535677085696 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚482 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42670365435904 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 532495206239 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43732112626688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13849180 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚44267593534464 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 532687202771 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚50 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231641088 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 555634618880 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13871776 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42670348396544 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚02ð÷3%0æ + _wandbruntime‚598  _runtime‚598~xtúÏ&ç ÉM O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11158488 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 536475489792 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10322443 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 536279298560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚44333644192768 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚535 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 533289089187 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚43267236870307 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13892674 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 _step‚51 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 536475489792 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚44149692305408 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚43797168702976 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42733964820480 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25051162 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3484162 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚44706135033856 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13892674 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11158488 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237919232 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 536279298560 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 533481085927 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚43267445906407 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚43797168702976 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6837746 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11158486 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13870052 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11158486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚44333644192768 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43797151663616 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3484694 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 556467632128 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚43797151663616 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10321908 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚44706397856768 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262822912 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚42733947781120 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 556705551360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚44149667401728 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0>Cè ÍM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚532 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 _step‚51 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 536475489792 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚44149692305408 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚43797168702976 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42733964820480 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25051162 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3484162 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚44706135033856 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13892674 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11158488 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237919232 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 536279298560 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 533481085927 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚43267445906407 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚43797168702976 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6837746 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11158486 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13870052 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11158486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚44333644192768 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43797151663616 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3484694 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 556467632128 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚43797151663616 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10321908 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚44706397856768 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262822912 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚42733947781120 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 556705551360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚44149667401728 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11158488 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 536475489792 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10322443 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 536279298560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚44333644192768 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚535 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 533289089187 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚43267236870307 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13892674 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47958496166400 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 575624453120 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚46984081512448 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14896132 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47559492748288 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47559705965568 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11967968 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3737235 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 598054541312 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚45850317625344 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚46422743186883 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚46422534150991 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788389884309909e+09 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚518 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11967966 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚46984064473088 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7329412 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221155328 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26886696 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 575428275200l’™—&$÷ ÂO TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14896132 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14918728 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47958496166400 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 575624453120 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚46984081512448 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14896132 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47559492748288 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47559705965568 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11967968 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3737235 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 598054541312 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚45850317625344 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚46422743186883 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚46422534150991 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788389884309909e+09 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚518 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11967966 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚46984064473088 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7329412 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221155328 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26886696 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStª=x#¢atsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 575428275200 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚46984064473088 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11967966 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚47958250107392 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚458503346647046½å†(0ø + _wandbruntime‚640  _runtime‚640áh Ï&ù ÉM EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26924114 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11984486 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47625330176000 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚47049103510016 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3742311 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚55 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7339449 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14939626 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 598904902144 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚46487136454615 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 trainer/global_step‚699 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14939626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7339446 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚48024872023552 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11984488 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3742835 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 573202405335 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11984486 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚47049120549376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26901490 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚46486927418515 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚47049120549376 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚26901490 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚47049103510016 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 576226665984 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11081757 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47425942217728 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47625543406592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11082284 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _runtime‚ 640.821528533 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14917004 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 576422857216 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 598669080064 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚45913917009920 % _timestamp‚1.7883898850648408e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0ÞèD(Ó&ú ÍM @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 573010408595 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11984486 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47625330176000 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚47049103510016 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3742311 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚55 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7339449 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14939626 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 598904902144 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚46487136454615 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 trainer/global_step‚699 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14939626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7339446 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚48024872023552 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11984488 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3742835 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 573202405335 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11984486 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚47049120549376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26901490 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚46486927418515 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚47049120549376 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚26901490 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚47049103510016 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 576226665984 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11081757 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47425942217728 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47625543406592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11082284 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _runtime‚ 640.821528533 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14917004 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 576422857216 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 598669080064 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚45913917009920 % _timestamp‚1.7883898850648408e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26924114 FDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚50851391962112 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 615375642624 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚50636691537408 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚749 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚50851391962112 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11827834 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 615571820544 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883899264873636e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 615375642624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚50851605179392 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚49642224699199 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28759648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7831115 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 615571820544 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11828329 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260739072 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚50236016319488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0büý¼É'‰ ÃO XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260739072 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚50236016319488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚15943084 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚50851391962112 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 615375642624 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚50636691537408 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚749 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚50851391962112 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11827834 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 615571820544 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883899264873636e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 615375642624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚50851605179392 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚49642224699199 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28759648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7831115 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 615571820544 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11828329 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360:“ïx0Š + _wandbruntime‚682  _runtime‚682ÁHÆáÏ&‹ ÉM NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12810486 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 640691021824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 616370224640 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _runtime‚ 682.998314831 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚28797066 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 616174033408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚749 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15963956 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚50917229389824 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚50702217033728 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 616174033408 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28797066 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4002236 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269114368 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚49706617966723 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚51342908055552 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 612923724743 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 612731728003 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚50301072395776 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7841146 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12810486 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11842870 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15963956 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 640935232512 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705•»2ÝrŒ ÍM ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 616174033408 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28797066 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4002236 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269114368 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚49706617966723 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚51342908055552 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 612923724743 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 612731728003 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚50301072395776 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7841146 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12810486 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11842870 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15963956 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 640935232512 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12810486 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 640691021824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 616370224640 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _runtime‚ 682.998314831 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚28797066 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 616174033408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚749 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15963956 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚50917229389824 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚50702217033728 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243;é#ñ½0 + _wandbruntime‚682  _runtime‚682ltáRŽ Ì 1loss/SRqqt3/assignment_loss‚2.063584016705001 2loss/SRqqt4/assignment_loss‚2.6523992047793623 /loss/FBt4/detection_loss‚3.1659136259037517 1loss/FBt1/assignment_loss‚0.48086519055068494 1loss/FBt2/assignment_loss‚0.24920085531536695 /loss/FRt4/detection_loss‚2.7718984772052084 1loss/SRqqt2/detection_loss‚1.3737654413218083 _step‚60 /loss/FRt4/assignment_loss‚5.219028800725937 0loss/FBt3/assignment_loss‚0.7738988987513862 /loss/FRt3/detection_loss‚2.6555876425902047 1loss/SRqqt1/detection_loss‚1.7745909612211916 .loss/FBt3/detection_loss‚2.647860199213028 %loss/total_loss‚52.47727055543349 _runtime‚ 682.999553335 trainer/global_step‚749 0loss/FRt3/assignment_loss‚3.5980490016937257 1loss/SRqqt1/assignment_loss‚1.870781666840668 0loss/SRqqt3/detection_loss‚1.835922598010964 0loss/FBt4/assignment_loss‚0.3752337742044584 2loss/SRqqt2/assignment_loss‚1.8497422977472129 .loss/FRt2/detection_loss‚1.867630099553777 /loss/FBt2/detection_loss‚0.8138010982204886 epoch‚1 $ _timestamp‚1.788389927243721e+09 /loss/FRt2/assignment_loss‚2.173810198903084 /loss/FRt1/detection_loss‚2.4635748208190003 /loss/SRqqt4/detection_loss‚2.52284705725269 .loss/FBt1/detection_loss‚3.174358230829239 /loss/FRt1/assignment_loss‚4.102926397075255*“Ó»V Ð 2loss/SRqqt2/assignment_loss‚1.8497422977472129 .loss/FRt2/detection_loss‚1.867630099553777 /loss/FBt2/detection_loss‚0.8138010982204886 epoch‚1 $ _timestamp‚1.788389927243721e+09 /loss/FRt2/assignment_loss‚2.173810198903084 /loss/FRt1/detection_loss‚2.4635748208190003 /loss/SRqqt4/detection_loss‚2.52284705725269 .loss/FBt1/detection_loss‚3.174358230829239 /loss/FRt1/assignment_loss‚4.102926397075255 1loss/SRqqt3/assignment_loss‚2.063584016705001 2loss/SRqqt4/assignment_loss‚2.6523992047793623 /loss/FBt4/detection_loss‚3.1659136259037517 1loss/FBt1/assignment_loss‚0.48086519055068494 1loss/FBt2/assignment_loss‚0.24920085531536695 /loss/FRt4/detection_loss‚2.7718984772052084 1loss/SRqqt2/detection_loss‚1.3737654413218083 _step‚60 /loss/FRt4/assignment_loss‚5.219028800725937 0loss/FBt3/assignment_loss‚0.7738988987513862 /loss/FRt3/detection_loss‚2.6555876425902047 1loss/SRqqt1/detection_loss‚1.7745909612211916 .loss/FBt3/detection_loss‚2.647860199213028 %loss/total_loss‚52.47727055543349 _runtime‚ 682.999553335 trainer/global_step‚749 0loss/FRt3/assignment_loss‚3.5980490016937257 1loss/SRqqt1/assignment_loss‚1.870781666840668 0loss/SRqqt3/detection_loss‚1.835922598010964 0loss/FBt4/assignment_loss‚0.3752337742044584<Ég+' ‚(:’ ®ÔâÔõÃÅu gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.608' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.321600000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Gä.1‘ ‚(:¦ ®ÔâÔˆªìx disk.nvme0n1.in‚ 39.72265625 disk.sda1.in‚0$ memory_percent‚83.36999416351318) proc.memory.availableMB‚ 2724.66015625 disk.rbd3.out‚ 1.28515625 disk.sda3.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 2902.39453125 disk.sda.in‚0 disk.sda1.out‚0 disk.sda2.out‚0 proc.cpu.threads‚17 disk.dm-1.in‚0 network.recv‚296255" disk.dm-0.out‚3487.07861328125* proc.memory.percent‚17.714810371398926 disk.rbd1.out‚ 0.306640625% disk.nvme0n1.out‚3487.07861328125 disk.sda.out‚ 28.2734375 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0) disk./.usagePercent‚69.97228544356724 disk.dm-0.in‚ 39.72265625 disk.sda3.out‚ 28.2734375 disk.rbd2.out‚29.94970703125' disk.nvme0c0n1.out‚3487.07861328125 cpu‚23.442712687675417% disk./.usageGB‚1042.6376190185547 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 39.72265625 disk.dm-1.out‚ 28.2734375 network.sent‚ 1.149503e+06 disk.rbd2.in‚0 disk.drbd1777.in‚0×Î8õ‘’ ‚(:† ½ÔâÔ¢À¶u gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚93.893' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7786 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0®|.“ ‚(:£ ½ÔâÔõ•Ãx% disk./.usageGB‚1042.6177368164062 disk.dm-1.out‚ 29.23828125 disk.rbd3.out‚ 1.28515625# proc.memory.rssMB‚ 2902.39453125" disk.dm-0.out‚3488.42529296875 disk.sda.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0% disk.nvme0n1.out‚3488.42529296875 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚ 39.7578125 disk.dm-0.in‚ 39.7578125 proc.cpu.threads‚17 disk.sda1.out‚0 network.recv‚301020 disk.rbd2.in‚0 disk.sda.out‚ 29.23828125 disk.rbd2.out‚31.00048828125 disk.nvme0n1.in‚ 39.7578125 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.sda3.out‚ 29.23828125 disk.dm-1.in‚0( proc.memory.availableMB‚ 2724.4921875* proc.memory.percent‚17.714810371398926 network.sent‚ 1.235751e+06 disk.rbd1.out‚ 0.306640625' disk.nvme0c0n1.out‚3488.42529296875 cpu‚23.48425641291316 disk.rbd1.in‚0 disk.sda2.out‚0 disk.sda3.in‚0$ memory_percent‚83.37101936340332( disk./.usagePercent‚69.9709511322988n¾U¨œ” ‚(:‘ ÌÔâÔÐÁŒv gpu.0.gpu‚10.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.164' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.832800000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*×i • ‚(:• ÌÔâÔµääw disk.sda3.in‚0 disk.sda1.in‚0 disk.nvme0c0n1.in‚ 42.203125' proc.memory.availableMB‚ 2724.234375 disk.drbd1777.out‚0 network.recv‚305731 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.306640625 disk.rbd2.in‚0 disk.sda.in‚0# proc.memory.rssMB‚ 2902.39453125 disk.sda2.out‚0 disk.rbd3.in‚0 disk.sda3.out‚29.5625' disk.nvme0c0n1.out‚3592.56689453125* proc.memory.percent‚17.714810371398926 disk.rbd3.out‚ 1.34765625 disk.dm-0.in‚ 42.203125 disk.dm-1.out‚29.5625 disk.rbd0.out‚0.078125 proc.cpu.threads‚17) disk./.usagePercent‚69.97103740683784 disk.dm-1.in‚0 disk.nvme0n1.in‚ 42.203125% disk.nvme0n1.out‚3592.56689453125 disk.rbd0.in‚0 cpu‚23.523100048475975 network.sent‚ 1.241889e+06% disk./.usageGB‚1042.6190223693848 disk.rbd2.out‚32.15283203125 disk.sda1.out‚0$ memory_percent‚83.37259292602539" disk.dm-0.out‚3592.56689453125 disk.sda.out‚29.5625 disk.drbd1777.in‚0ÂÏ(0– + _wandbruntime‚724  _runtime‚724qzƒÀ— º $ lr-AdamW/pg1‚0.00171314072499137 $ lr-AdamW/pg2‚0.00171314072499137 trainer/global_step‚799 % _timestamp‚1.7883899682729304e+09 _runtime‚ 724.029982454 _step‚61àäÜõĘ ¾ $ lr-AdamW/pg1‚0.00171314072499137 $ lr-AdamW/pg2‚0.00171314072499137 trainer/global_step‚799 % _timestamp‚1.7883899682729304e+09 _runtime‚ 724.029982454 _step‚61=®¥0™ + _wandbruntime‚724  _runtime‚724ƒiº¤Å'š ¿O [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚53487968165888 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 724.030096599 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53487968165888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 % _timestamp‚1.7883899682737021e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13619966 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54143291175936 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚16990036 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 682055499264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚52210256082944 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚54143504393216 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚30632600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 651851160995 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 682293431808 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13619968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 655519187968 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237932544 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚54143504393216 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 655323010048 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 655323010048 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚16990036 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚54595021852672 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53912991257088 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8332812 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚52862124283299 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚52861915247407 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚52210273122304 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53912966353408 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13619968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4256770 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 trainer/global_step‚799EvªÉ'› ÃO L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚16990036 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚54595021852672 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53912991257088 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8332812 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚52862124283299 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚52861915247407 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚52210273122304 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53912966353408 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13619968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4256770 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 trainer/global_step‚799 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚53487968165888 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 724.030096599 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53487968165888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 % _timestamp‚1.7883899682737021e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13619966 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54143291175936 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚16990036 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 682055499264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚52210256082944 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚54143504393216 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚30632600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 651851160995 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 682293431808 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13619968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 655519187968 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237932544 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚54143504393216 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 655323010048 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 655323010048 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0>¢Žð²0œ + _wandbruntime‚724  _runtime‚724̉Ðk^ ÉM N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚54209341834240 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚53978516753408 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 682911177728 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚53553007202816 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 683142805504 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚30670018 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13636486 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚30647394 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚506 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 656317592064 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17033530 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚52273872506880 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 652453047411 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚54209128603648 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256531456 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4262358 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53978491849728 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8342849 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 652645044151 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 656317592064 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54209341834240 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12604701 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚799 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 _runtime‚ 724.790346959 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 656121400832 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚30647394 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17010908 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚30670018 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13636488 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 656121400832 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _step‚63 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚54661659558912 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4261855 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231627776 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚52273855467520 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚53553007202816 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232<=Ó&ž ÍM ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8342849 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 652645044151 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 656317592064 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54209341834240 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12604701 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚799 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 _runtime‚ 724.790346959 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 656121400832 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚30647394 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17010908 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚30670018 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13636488 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 656121400832 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _step‚63 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚54661659558912 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4261855 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231627776 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚52273855467520 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚53553007202816 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚54209341834240 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚53978516753408 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 682911177728 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚53553007202816 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 683142805504 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚30670018 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13636486 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚30647394 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚506 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 656317592064 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17033530 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚52273872506880 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 652453047411 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚54209128603648 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256531456 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4262358 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53978491849728 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0? “{ 0Ÿ + _wandbruntime‚724  _runtime‚724ÓªU  Ï /loss/FRt4/assignment_loss‚5.276542149338067 2loss/SRqqt2/assignment_loss‚1.5498699164667795 0loss/SRqqt1/detection_loss‚1.732987891882658 1loss/SRqqt2/detection_loss‚1.3483433537473974 0loss/SRqqt4/detection_loss‚2.822329520168951 /loss/FBt3/detection_loss‚2.0367611035704614 .loss/FBt4/detection_loss‚4.045620673894883 _runtime‚ 724.791559935 0loss/FBt3/assignment_loss‚0.5244982197077479 %loss/total_loss‚53.32044103474657 $ _timestamp‚1.788389969035749e+09 /loss/FRt3/assignment_loss‚4.105894975466271 0loss/FBt1/assignment_loss‚0.7611258183088567 /loss/FRt3/detection_loss‚2.6147343451029634 epoch‚1 /loss/FRt2/detection_loss‚2.1595985551354704 /loss/FRt4/detection_loss‚2.6851822745566274 trainer/global_step‚799 _step‚64 0loss/FBt4/assignment_loss‚0.3240593763772267 .loss/FRt1/assignment_loss‚3.93376130490076 2loss/SRqqt1/assignment_loss‚1.8248726923816971 2loss/SRqqt3/assignment_loss‚1.7205925444045018 1loss/SRqqt3/detection_loss‚1.9749001786880886 1loss/FBt2/assignment_loss‚0.23584302437224222 /loss/FRt1/detection_loss‚2.3064636957077753 /loss/FBt1/detection_loss‚2.9552501373820834 /loss/FBt2/detection_loss‚0.7726496981785578 0loss/FRt2/assignment_loss‚3.2196434845206556 2loss/SRqqt4/assignment_loss‚2.3889161004858503”©‰Y¡ Ó $ _timestamp‚1.788389969035749e+09 /loss/FRt3/assignment_loss‚4.105894975466271 0loss/FBt1/assignment_loss‚0.7611258183088567 /loss/FRt3/detection_loss‚2.6147343451029634 epoch‚1 /loss/FRt2/detection_loss‚2.1595985551354704 /loss/FRt4/detection_loss‚2.6851822745566274 trainer/global_step‚799 _step‚64 0loss/FBt4/assignment_loss‚0.3240593763772267 .loss/FRt1/assignment_loss‚3.93376130490076 2loss/SRqqt1/assignment_loss‚1.8248726923816971 2loss/SRqqt3/assignment_loss‚1.7205925444045018 1loss/SRqqt3/detection_loss‚1.9749001786880886 1loss/FBt2/assignment_loss‚0.23584302437224222 /loss/FRt1/detection_loss‚2.3064636957077753 /loss/FBt1/detection_loss‚2.9552501373820834 /loss/FBt2/detection_loss‚0.7726496981785578 0loss/FRt2/assignment_loss‚3.2196434845206556 2loss/SRqqt4/assignment_loss‚2.3889161004858503 /loss/FRt4/assignment_loss‚5.276542149338067 2loss/SRqqt2/assignment_loss‚1.5498699164667795 0loss/SRqqt1/detection_loss‚1.732987891882658 1loss/SRqqt2/detection_loss‚1.3483433537473974 0loss/SRqqt4/detection_loss‚2.822329520168951 /loss/FBt3/detection_loss‚2.0367611035704614 .loss/FBt4/detection_loss‚4.045620673894883 _runtime‚ 724.791559935 0loss/FBt3/assignment_loss‚0.5244982197077479 %loss/total_loss‚53.32044103474657@«40Äœ¢ ‚(:‘ ÛÔâÔ½ã¦u gpu.0.gpu‚21.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.693' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.938599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Åí‚™0£ ‚(:¥ ÛÔâÔíÃŒx disk.sda2.out‚0 disk.rbd1.in‚0# proc.memory.rssMB‚ 2902.39453125 disk.dm-1.out‚ 30.30078125) disk./.usagePercent‚69.97107094382483 disk.nvme0n1.in‚ 42.6484375 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd3.out‚ 1.34765625 disk.sda3.in‚0 disk.rbd1.out‚ 0.306640625 network.recv‚310256' disk.nvme0c0n1.out‚3691.66455078125" disk.dm-0.out‚3691.66455078125 disk.rbd2.out‚33.19580078125) proc.memory.availableMB‚ 2724.01171875 disk.sda.out‚ 30.30078125 disk.dm-1.in‚0* proc.memory.percent‚17.714810371398926 network.sent‚ 1.248309e+06 disk.sda.in‚0 disk.rbd0.out‚0.078125 disk.rbd2.in‚0$ memory_percent‚83.37395191192627% disk.nvme0n1.out‚3691.66455078125 disk.sda1.in‚0 disk.nvme0c0n1.in‚ 42.6484375 disk.sda2.in‚0 proc.cpu.threads‚17% disk./.usageGB‚1042.6195220947266 disk.sda1.out‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 42.6484375 disk.sda3.out‚ 30.30078125 cpu‚23.56138797162657fªv’¤ ‚(:‡ êÔâÔë³Óu gpu.0.gpu‚56.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.319' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.4638 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ü )¥ ‚(:ž êÔâÔõËx disk.dm-0.in‚43.40625 disk.dm-1.out‚ 30.76171875$ memory_percent‚83.37252140045166( disk./.usagePercent‚69.9710850242392 disk.rbd2.out‚34.34814453125 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚3694.34814453125 disk.sda2.out‚0 disk.sda3.in‚0* proc.memory.percent‚17.714810371398926 disk.nvme0n1.in‚43.40625 disk.rbd0.out‚0.078125# proc.memory.rssMB‚ 2902.39453125 network.recv‚315604 disk.rbd1.out‚ 0.306640625 disk.sda3.out‚ 30.76171875 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.sda1.in‚0 cpu‚23.59805745963996 disk.sda.in‚0 disk.sda.out‚ 30.76171875 disk.drbd1777.in‚0" disk.dm-0.out‚3694.34814453125) proc.memory.availableMB‚ 2724.24609375 proc.cpu.threads‚17% disk.nvme0n1.out‚3694.34814453125 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.nvme0c0n1.in‚43.40625 disk.sda2.in‚0 network.sent‚ 1.334577e+06% disk./.usageGB‚1042.6197319030762 disk.rbd2.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0n5eEœ¦ ‚(:‘ ùÔâÔ÷‹Àu gpu.0.gpu‚60.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚151.44' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.287999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¹ €l § ‚(:• ùÔâÔëÌw disk.rbd0.in‚0 disk.sda3.out‚31.53125 disk.rbd2.in‚0( disk./.usagePercent‚69.9710850242392 disk.drbd1777.in‚0 disk.dm-1.out‚31.53125 network.sent‚ 1.340055e+06 disk.drbd1777.out‚0% disk.nvme0n1.out‚3792.64111328125 cpu‚23.63406368613376 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.out‚ 0.306640625 disk.nvme0n1.in‚43.71875* proc.memory.percent‚17.714810371398926 network.recv‚319380 disk.rbd0.out‚0.078125' disk.nvme0c0n1.out‚3792.64111328125 disk.sda2.out‚0 disk.dm-0.in‚43.71875" disk.dm-0.out‚3792.64111328125 disk.dm-1.in‚0 disk.rbd2.out‚35.39501953125 disk.sda1.out‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda.out‚31.53125$ memory_percent‚83.37266445159912) proc.memory.availableMB‚ 2724.22265625# proc.memory.rssMB‚ 2902.39453125 proc.cpu.threads‚17% disk./.usageGB‚1042.6197319030762 disk.nvme0c0n1.in‚43.71875 disk.sda3.in‚0 disk.sda1.in‚0È00¨ + _wandbruntime‚766  _runtime‚766™ìBsé ½ & lr-AdamW/pg2‚0.0017099923215196388 trainer/global_step‚849 $ _timestamp‚1.788390010540119e+09 _runtime‚ 766.296126595 _step‚65 & lr-AdamW/pg1‚0.0017099923215196388@ýzæǪ Á & lr-AdamW/pg1‚0.0017099923215196388 & lr-AdamW/pg2‚0.0017099923215196388 trainer/global_step‚849 $ _timestamp‚1.788390010540119e+09 _runtime‚ 766.296126595 _step‚65Ag »¤0« + _wandbruntime‚766  _runtime‚766œCËhA¬ ¿O UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚57435403607040 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231641088 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14445966 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚57913646252544 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8834512 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 695466555392 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14445968 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256544768 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14445966 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13349514 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13349013 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56081605795615 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚849 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 724380179456 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 % _timestamp‚1.7883900105411768e+09 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚57435403607040 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚32482954 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14445968 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚18036988 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚56739920012288 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚32505552 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18059584 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57189241169408 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57435190389760 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4514999 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚498 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚R­ÙD„55390225311744 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18059584 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 695270377472 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55390242351104 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 691572480403 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56739920012288 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4514501 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚57913389707776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _runtime‚ 766.297425483 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56739937051648 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57435190389760 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚18036988 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57189266073088 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 695466555392 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 695270377472 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚501 _step‚66Å{VÉ'­ ÃO _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚56739937051648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 724148538368 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 691380483871 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8834515 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56739937051648 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57435190389760 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚18036988 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57189266073088 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 695466555392 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 695270377472 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚501 _step‚66 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚57435403607040 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231641088 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14445966 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚57913646252544 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8834512 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 695466555392 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14445968 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256544768 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14445966 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13349514 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13349013 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56081605795615 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚849 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 724380179456 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 % _timestamp‚1.7883900105411768e+09 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚57435403607040 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚32482954 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14445968 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚18036988 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚56739920012288 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚32505552 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18059584 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57189241169408 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57435190389760 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4514999 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚498 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚55390225311744 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18059584 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 695270377472 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55390242351104 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 691572480403 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56739920012288 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4514501 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚57913389707776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _runtime‚ 766.297425483 IDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32542970 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14462486 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14462488 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57254766665728 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18080482 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚56145999063139 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13364642 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 725234062848 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 692366363559 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚32542970 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4520093 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8844549 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚57980025632256 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚57979764906496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8844546 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚32520346 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13364142 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18057860 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235822080 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18080482 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57501027817472 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725760 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 696264959488 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚849 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4519596 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56804959049216 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 696068768256 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚57501241048064 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56146208099239 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13364142 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18057860 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235822080 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18080482 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57501027817472 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725760 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 696264959488 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚849 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4519596 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56804959049216 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 696068768256 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚57501241048064 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56146208099239 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32542970 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14462486 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14462488 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57254766665728 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18080482 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚56145999063139 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚åp*?B 10869712896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13364642 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 725234062848 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 692366363559 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚32542970 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4520093 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8844549 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚57980025632256 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚57979764906496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8844546 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0CÜR£ 0± + _wandbruntime‚767  _runtime‚767¿Qq~S² Í 1loss/SRqqt4/detection_loss‚2.4851726539551264 %loss/total_loss‚51.96347063914912 /loss/FRt1/assignment_loss‚3.930238568534454 /loss/FRt4/assignment_loss‚4.562342972755432 2loss/SRqqt2/assignment_loss‚1.6014071245278632 0loss/FBt4/assignment_loss‚0.6120547772586886 /loss/FRt2/detection_loss‚1.7384516093709053 /loss/FBt1/detection_loss‚3.7442510468619212 0loss/FRt3/assignment_loss‚3.9422308794863814 .loss/FRt4/detection_loss‚2.582462402979533 $ _timestamp‚1.788390011310155e+09 _step‚68 0loss/FBt1/assignment_loss‚1.2947067724806922 0loss/FBt3/assignment_loss‚0.7584839931324773 /loss/FRt3/detection_loss‚2.3723031646245487 /loss/FBt3/detection_loss‚2.4407517028891523 0loss/SRqqt4/assignment_loss‚2.50079089497763 1loss/FBt2/assignment_loss‚0.27171233020777014 0loss/SRqqt3/detection_loss‚1.841013766599424 trainer/global_step‚849 2loss/SRqqt3/assignment_loss‚2.0361836811508796 1loss/SRqqt1/detection_loss‚1.6488191157836338 /loss/FRt2/assignment_loss‚2.440665973435848 _runtime‚ 767.066310487 /loss/FRt1/detection_loss‚2.3919850795840225 1loss/SRqqt2/detection_loss‚1.2136699718480208 /loss/FBt2/detection_loss‚0.6819599679423661 .loss/FBt4/detection_loss‚2.829066534837087 epoch‚1 1loss/SRqqt1/assignment_loss‚2.042745653925271&”W³ Ñ 2loss/SRqqt3/assignment_loss‚2.0361836811508796 1loss/SRqqt1/detection_loss‚1.6488191157836338 /loss/FRt2/assignment_loss‚2.440665973435848 _runtime‚ 767.066310487 /loss/FRt1/detection_loss‚2.3919850795840225 1loss/SRqqt2/detection_loss‚1.2136699718480208 /loss/FBt2/detection_loss‚0.6819599679423661 .loss/FBt4/detection_loss‚2.829066534837087 epoch‚1 1loss/SRqqt1/assignment_loss‚2.042745653925271 1loss/SRqqt4/detection_loss‚2.4851726539551264 %loss/total_loss‚51.96347063914912 /loss/FRt1/assignment_loss‚3.930238568534454 /loss/FRt4/assignment_loss‚4.562342972755432 2loss/SRqqt2/assignment_loss‚1.6014071245278632 0loss/FBt4/assignment_loss‚0.6120547772586886 /loss/FRt2/detection_loss‚1.7384516093709053 /loss/FBt1/detection_loss‚3.7442510468619212 0loss/FRt3/assignment_loss‚3.9422308794863814 .loss/FRt4/detection_loss‚2.582462402979533 $ _timestamp‚1.788390011310155e+09 _step‚68 0loss/FBt1/assignment_loss‚1.2947067724806922 0loss/FBt3/assignment_loss‚0.7584839931324773 /loss/FRt3/detection_loss‚2.3723031646245487 /loss/FBt3/detection_loss‚2.4407517028891523 0loss/SRqqt4/assignment_loss‚2.50079089497763 1loss/FBt2/assignment_loss‚0.27171233020777014 0loss/SRqqt3/detection_loss‚1.841013766599424 trainer/global_step‚849D[¤'¢›´ ‚(: ˆÕâÔê‹Âv gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.8' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.560000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0j’ ‚(:“ ˆÕâÔÉä©x$ memory_percent‚83.38298797607422 cpu‚23.66821668710518 disk.rbd1.in‚0 disk.sda1.out‚0 disk.nvme0c0n1.in‚44.4375 disk.sda2.out‚0 disk.dm-0.in‚44.4375 disk.dm-0.out‚3891.576171875 disk.sda3.out‚ 31.91796875 disk.drbd1777.in‚0 network.recv‚324254 disk.rbd3.out‚ 1.36328125 disk.dm-1.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚36.54736328125 disk.sda.in‚0* proc.memory.percent‚17.714810371398926 proc.cpu.threads‚17 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0% disk.nvme0c0n1.out‚3891.576171875# proc.memory.rssMB‚ 2902.39453125 disk.rbd0.in‚0 network.sent‚ 1.346176e+06) disk./.usagePercent‚69.97108067211111 disk.nvme0n1.in‚44.4375# disk.nvme0n1.out‚3891.576171875 disk.sda.out‚ 31.91796875% disk./.usageGB‚1042.6196670532227 disk.sda2.in‚0& proc.memory.availableMB‚ 2722.53125 disk.rbd3.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 31.91796875 disk.rbd1.out‚ 0.306640625»Xú ‚(:† —ÕâÔ–§w gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚94.957' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.9914 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08 N(· ‚(: —ÕâÔ¥ÑÆy disk.rbd1.out‚ 0.306640625 network.sent‚ 1.433911e+06 disk.dm-0.in‚44.4375 proc.cpu.threads‚17 network.recv‚330587) disk./.usagePercent‚69.97105916747824 disk.sda2.out‚0 disk.sda.in‚0 cpu‚23.701478446206252 disk.nvme0c0n1.in‚44.4375 disk.sda3.in‚0 disk.rbd2.out‚37.59423828125 disk.nvme0n1.in‚44.4375 disk.dm-1.in‚0 disk.rbd2.in‚0) proc.memory.availableMB‚ 2437.47265625% disk.nvme0n1.out‚3892.09033203125 disk.sda1.in‚0 disk.drbd1777.in‚0' disk.nvme0c0n1.out‚3892.09033203125 disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0 disk.sda3.out‚ 32.55078125% disk./.usageGB‚1042.6193466186523 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0" disk.dm-0.out‚3892.09033203125$ memory_percent‚85.12284755706787# proc.memory.rssMB‚ 2902.39453125 disk.sda1.out‚0 disk.dm-1.out‚ 32.55078125* proc.memory.percent‚17.714810371398926 disk.rbd0.out‚0.078125 disk.sda.out‚ 32.55078125 wàØ0¸ + _wandbruntime‚808  _runtime‚808²‰BĹ ¾ & lr-AdamW/pg2‚0.0017061336450485004 trainer/global_step‚899 % _timestamp‚1.7883900526233075e+09 _runtime‚ 808.379276395 _step‚69 & lr-AdamW/pg1‚0.0017061336450485004·uOðȺ  _step‚69 & lr-AdamW/pg1‚0.0017061336450485004 & lr-AdamW/pg2‚0.0017061336450485004 trainer/global_step‚899 % _timestamp‚1.7883900526233075e+09 _runtime‚ 808.379276395E£jkc0» + _wandbruntime‚808  _runtime‚8089–à@Ä'¼ ¾O ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 735413922816 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚61231905473536 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚59301505379715 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 735217744896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19083940 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 731293799811 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34378504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 $ _timestamp‚1.788390052624369e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59991871858688 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14109696 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19083940 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4774002 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34355906 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14110217 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚19106536 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚60727089603584 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚60465540889088 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 808.380571094 _step‚70 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15271968 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60727089603584 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚60727302820864 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15271966 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9336215 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚15271968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4773484 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 766111971840 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15271966 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚521 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚58570194540544 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19106536 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚518 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9336212 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 731101803279 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚59991871858688 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚59991888898048 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚58570211579904 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 735217744896 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚60727302820864 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252612608 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚59991888898048 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚58570211579904 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 735217744896 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚60727302820864 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252612608 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚59301505379715 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 735217744896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19083940 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 731293799811 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34378504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 $ _timestamp‚1.788390052624369e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59991871858688 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14109696 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19083940 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4774002 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34355906 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14110217 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚19106536 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚60727089603584 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚60465540889088 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 808.380571094 _step‚70 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15271968 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60727089603584 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚60727302820864FÝúbÔ0¾ + _wandbruntime‚809  _runtime‚809#¶ÌJZ¿ ÈM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚61298261952512 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚60793140261888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34415922 _step‚71 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚60056927934976 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚60056910895616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15288486 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61297990740992 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60793140261888 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 766949259264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 736016135680 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 767195567104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246307840 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 736212326912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271211520 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15288488 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19127434 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚59365898647447 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚34415922 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚508 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚505 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚58633810964480 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60792927031296 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚19127434 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9346246 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60531066385408 M=DeviceStatsMonitor.on_train_batch_end/reserved_bIœ†%tytes.all.peak‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14124991 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚60531041481728 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚60056910895616 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚60056927934976 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 736212326912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15288486 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19104812 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4778745 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4779250 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 732087682967 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15288488 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 $ _timestamp‚1.788390053607099e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19104812 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚58633793925120 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824‚ëÒ&À ÌM S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚60793140261888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34415922 _step‚71 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚60056927934976 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚60056910895616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15288486 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61297990740992 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60793140261888 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 766949259264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 736016135680 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 767195567104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246307840 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 736212326912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271211520 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15288488 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19127434 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚59365898647447 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚34415922 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚508 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚505 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚58633810964480 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60792927031296 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚19127434 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9346246 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60531066385408 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14124991 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚60531041481728 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚60056910895616 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚60056927934976 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 736212326912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15288486 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19104812 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4778745 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4779250 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 732087682967 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15288488 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 $ _timestamp‚1.788390053607099e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19104812 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚58633793925120 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚61298261952512 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705GfÄ® 0Á + _wandbruntime‚809  _runtime‚809êâxR Ì 0loss/FBt3/assignment_loss‚0.4960612053020365 1loss/SRqqt3/assignment_loss‚1.953260588184507 _step‚72 .loss/FBt3/detection_loss‚2.417676281183958 %loss/total_loss‚51.33215801806265 /loss/FBt4/detection_loss‚2.7968139913347034 /loss/FRt3/assignment_loss‚4.020533391686737 /loss/FRt4/assignment_loss‚4.526927840882453 2loss/SRqqt2/assignment_loss‚1.5375995460069842 1loss/FBt1/assignment_loss‚0.28845157374598784 0loss/FBt2/assignment_loss‚0.2296087102775355 .loss/FRt4/detection_loss‚2.625739602075107 1loss/SRqqt4/detection_loss‚2.5048585482265637 0loss/FBt4/assignment_loss‚0.6518834133748391 1loss/SRqqt1/detection_loss‚1.8878967854109678 1loss/SRqqt2/detection_loss‚1.1462515908189945 epoch‚1 % _timestamp‚1.7883900536096017e+09 _runtime‚ 809.365422791 /loss/FRt3/detection_loss‚2.8755002920744848 -loss/FRt2/detection_loss‚2.12498350036533 .loss/FBt1/detection_loss‚3.488616636821202 trainer/global_step‚899 /loss/FRt1/detection_loss‚2.1997082397904038 0loss/FRt1/assignment_loss‚3.8258371296918616 1loss/SRqqt1/assignment_loss‚2.061436946486885 /loss/FRt2/assignment_loss‚2.395722858931707 2loss/SRqqt4/assignment_loss‚2.3618331482444983 0loss/SRqqt3/detection_loss‚2.093167816373435 /loss/FBt2/detection_loss‚0.8217883807714699!o¶Và Ð 0loss/FBt3/assignment_loss‚0.4960612053020365 1loss/SRqqt3/assignment_loss‚1.953260588184507 _step‚72 .loss/FBt3/detection_loss‚2.417676281183958 %loss/total_loss‚51.33215801806265 /loss/FBt4/detection_loss‚2.7968139913347034 /loss/FRt3/assignment_loss‚4.020533391686737 /loss/FRt4/assignment_loss‚4.526927840882453 2loss/SRqqt2/assignment_loss‚1.5375995460069842 1loss/FBt1/assignment_loss‚0.28845157374598784 0loss/FBt2/assignment_loss‚0.2296087102775355 .loss/FRt4/detection_loss‚2.625739602075107 1loss/SRqqt4/detection_loss‚2.5048585482265637 0loss/FBt4/assignment_loss‚0.6518834133748391 1loss/SRqqt1/detection_loss‚1.8878967854109678 1loss/SRqqt2/detection_loss‚1.1462515908189945 epoch‚1 % _timestamp‚1.7883900536096017e+09 _runtime‚ 809.365422791 /loss/FRt3/detection_loss‚2.8755002920744848 -loss/FRt2/detection_loss‚2.12498350036533 .loss/FBt1/detection_loss‚3.488616636821202 trainer/global_step‚899 /loss/FRt1/detection_loss‚2.1997082397904038 0loss/FRt1/assignment_loss‚3.8258371296918616 1loss/SRqqt1/assignment_loss‚2.061436946486885 /loss/FRt2/assignment_loss‚2.395722858931707 2loss/SRqqt4/assignment_loss‚2.3618331482444983 0loss/SRqqt3/detection_loss‚2.093167816373435 /loss/FBt2/detection_loss‚0.8217883807714699HcŒ&Ä ‚(:’ ¦ÕâÔõ÷¿u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.672' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.534399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0€™b®&Å ‚(:› ¦ÕâÔÿ³Žx network.recv‚334496% disk.nvme0n1.out‚3991.46923828125 disk.sda.in‚0 disk.drbd1777.out‚0$ memory_percent‚85.12513637542725# proc.memory.rssMB‚ 2902.39453125 disk.nvme0c0n1.in‚44.4375 disk.rbd3.in‚0" disk.dm-0.out‚3991.46923828125 disk.sda3.out‚ 32.83984375 proc.cpu.threads‚17 network.sent‚ 1.439097e+06$ disk./.usageGB‚1042.620475769043 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 32.83984375* proc.memory.percent‚17.714810371398926 disk.sda1.out‚0 disk.dm-1.in‚0 disk.nvme0n1.in‚44.4375' disk.nvme0c0n1.out‚3991.46923828125) disk./.usagePercent‚69.97113494570839 disk.sda.out‚ 32.83984375 disk.sda2.out‚0 disk.dm-0.in‚44.4375 disk.rbd1.out‚ 0.306640625 disk.rbd2.out‚38.74658203125) proc.memory.availableMB‚ 2437.09765625 cpu‚23.73270069957139b³I_Æ ‚(:’ µÕâÔý¯¹u gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚101.666' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚20.333199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ö°e&Ç ‚(:› µÕâÔ³›¢x disk.drbd1777.in‚0) disk./.usagePercent‚69.97113340966318' disk.nvme0c0n1.out‚4089.89111328125 disk.sda2.out‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚44.4375" disk.dm-0.out‚4089.89111328125 disk.sda3.in‚0% disk./.usageGB‚1042.6204528808594 disk.rbd0.in‚0 disk.sda3.out‚ 33.33203125 disk.rbd2.out‚39.80126953125 network.sent‚ 1.52543e+06% disk.nvme0n1.out‚4089.89111328125 disk.nvme0c0n1.in‚44.4375 disk.sda2.in‚0( proc.memory.availableMB‚ 2437.2578125* proc.memory.percent‚17.714810371398926 cpu‚23.763637351149566 disk.sda.out‚ 33.33203125 disk.sda1.in‚0 disk.dm-1.in‚0 disk.rbd2.in‚0$ memory_percent‚85.12415885925293 proc.cpu.threads‚17 disk.sda.in‚0 disk.rbd0.out‚0.078125# proc.memory.rssMB‚ 2902.39453125 disk.dm-1.out‚ 33.33203125 disk.rbd1.in‚0 disk.rbd1.out‚ 0.455078125 disk.drbd1777.out‚0 network.recv‚339614 disk.nvme0n1.in‚44.4375 disk.sda1.out‚0èömYÈ ‚(:’ ÄÕâÔ±ƒÇu gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.319' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.863799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0h|R!É ‚(:– ÄÕâÔÞµöx network.sent‚ 1.53156e+06 disk.sda.out‚ 33.88671875* proc.memory.percent‚17.714834213256836& disk.nvme0c0n1.out‚4090.7958984375( proc.memory.availableMB‚ 2437.0078125 disk.sda2.in‚0 disk.dm-0.in‚44.4375 disk.sda3.out‚ 33.88671875 disk.rbd1.out‚ 0.455078125 disk.rbd2.out‚41.03564453125 proc.cpu.threads‚17! disk.dm-0.out‚4090.7958984375 disk.dm-1.in‚0$ disk.nvme0n1.out‚4090.7958984375 disk.sda.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚44.4375 disk.sda3.in‚0 disk.dm-1.out‚ 33.88671875 disk.rbd2.in‚0 disk.nvme0n1.in‚44.4375 disk.rbd0.out‚0.078125" proc.memory.rssMB‚ 2902.3984375 cpu‚23.793829546100795 disk.rbd1.in‚0 network.recv‚344204$ disk./.usageGB‚1042.620475769043 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda1.in‚0) disk./.usagePercent‚69.97113494570839 disk.sda2.out‚0$ memory_percent‚85.12568473815918@m¼0Ê + _wandbruntime‚850  _runtime‚850Þª8çÂË ¼ % lr-AdamW/pg1‚0.001701567917306947 % lr-AdamW/pg2‚0.001701567917306947 trainer/global_step‚949 % _timestamp‚1.7883900948692474e+09 _runtime‚ 850.625229356 _step‚73mI`ÆÌ À trainer/global_step‚949 % _timestamp‚1.7883900948692474e+09 _runtime‚ 850.625229356 _step‚73 % lr-AdamW/pg1‚0.001701567917306947 % lr-AdamW/pg2‚0.001701567917306947I:¹sy0Í + _wandbruntime‚850  _runtime‚850Þ¨ëÄF&Î ¿O UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 770823122687 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚64018988817408 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚63243840744448 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 775165112320 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20153488 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚20130892 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9837915 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 771015119219 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚64019202034688 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚64549771503104 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚63741815705088 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚63243823705088 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14869129 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64550044825088 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 775361290240 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36228858 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16097968 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚64018988817408 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9837912 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 807980701696 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚62521195927923 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 775165112320 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚64019202034688 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36251456 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 808229120000 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5031217 YKDeviceStatsMonitor.on_train_batchýŠSœ_start/reserved_bytes.small_pool.allocated‚ 509607936 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16097968 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750øíSVÉ'Ï ÃO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚61750163769344 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚949 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚516 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 % _timestamp‚1.7883900948702939e+09 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚519 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5031733 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚62520986892031 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 770823122687 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚64018988817408 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚63243840744448 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 775165112320 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20153488 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚20130892 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9837915 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 771015119219 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚64019202034688 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚64549771503104 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚63741815705088 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚63243823705088 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14869129 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64550044825088 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 775361290240 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36228858 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16097968 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚64018988817408 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9837912 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 807980701696 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚62521195927923 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 775165112320 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚64019202034688 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36251456 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 808229120000 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5031217 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16097968 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14884387 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚63308862742016 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14884898 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9847949 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5036949 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚61813780193280 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚20151764 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5036441 $ _timestamp‚1.788390095621056e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20151764 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚61813763153920 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16114488 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242113536 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267017216 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚508 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚63308862742016 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _runtime‚ 851.377457145 _step‚75 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20174386 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚64085039475712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36288874 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚63308879781376 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9847946 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36266250 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚63807316297728 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 808821517824 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚63308879781376 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 775963503104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚62585589195655 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507j^þÖÒ&Ò ÌM QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9847946 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36266250 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚63807316297728 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 808821517824 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚63308879781376 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 775963503104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚62585589195655 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14884387 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚63308862742016 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14884898 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9847949 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5036949 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚61813780193280 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚20151764 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5036441 $ _timestamp‚1.788390095621056e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20151764 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚61813763153920 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16114488 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242113536 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267017216 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚508 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚63308862742016 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _runtime‚ 851.377457145 _step‚75 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20174386 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚64085039475712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36288874 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚63308879781376KTÊþ©0Ó + _wandbruntime‚851  _runtime‚851Y´nUÔ Ï 1loss/FBt3/assignment_loss‚0.46039188353375615 /loss/FRt3/assignment_loss‚4.041841337710251 1loss/SRqqt3/assignment_loss‚1.849874198077673 $ _timestamp‚1.788390095622828e+09 2loss/SRqqt2/assignment_loss‚1.5642321180356176 /loss/FBt1/detection_loss‚3.1640354990959167 /loss/FBt4/detection_loss‚3.6931647339037488 %loss/total_loss‚51.10132700706034 /loss/FBt3/detection_loss‚2.6811492180123047 0loss/FBt2/assignment_loss‚0.1702596510050974 0loss/FRt2/assignment_loss‚2.3917313494180377 /loss/FRt4/assignment_loss‚4.664931778845034 2loss/SRqqt4/assignment_loss‚2.4461795588334403 /loss/FBt1/assignment_loss‚0.408448718703137 /loss/FRt3/detection_loss‚2.5741841752588015 .loss/FRt2/detection_loss‚1.989563441668686 /loss/FRt1/assignment_loss‚3.662590914165851 2loss/SRqqt1/assignment_loss‚1.8802202977575697 1loss/SRqqt2/detection_loss‚1.1776047621230434 1loss/SRqqt4/detection_loss‚2.7148215925251997 _runtime‚ 851.378628533 _step‚76 trainer/global_step‚949 1loss/SRqqt1/detection_loss‚1.6735889296810906 1loss/SRqqt3/detection_loss‚2.0767746215843292 /loss/FBt2/detection_loss‚0.6850685721300053 epoch‚1 .loss/FRt4/detection_loss‚2.383121266176826 0loss/FBt4/assignment_loss‚0.3953142633703562 /loss/FRt1/detection_loss‚2.3522341254445696meˆ°+Õ Ó 1loss/SRqqt2/detection_loss‚1.1776047621230434 1loss/SRqqt4/detection_loss‚2.7148215925251997 _runtime‚ 851.378628533 _step‚76 trainer/global_step‚949 1loss/SRqqt1/detection_loss‚1.6735889296810906 1loss/SRqqt3/detection_loss‚2.0767746215843292 /loss/FBt2/detection_loss‚0.6850685721300053 epoch‚1 .loss/FRt4/detection_loss‚2.383121266176826 0loss/FBt4/assignment_loss‚0.3953142633703562 /loss/FRt1/detection_loss‚2.3522341254445696 1loss/FBt3/assignment_loss‚0.46039188353375615 /loss/FRt3/assignment_loss‚4.041841337710251 1loss/SRqqt3/assignment_loss‚1.849874198077673 $ _timestamp‚1.788390095622828e+09 2loss/SRqqt2/assignment_loss‚1.5642321180356176 /loss/FBt1/detection_loss‚3.1640354990959167 /loss/FBt4/detection_loss‚Ÿ‡k.3.6931647339037488 %loss/total_loss‚51.10132700706034 /loss/FBt3/detection_loss‚2.6811492180123047 0loss/FBt2/assignment_loss‚0.1702596510050974 0loss/FRt2/assignment_loss‚2.3917313494180377 /loss/FRt4/assignment_loss‚4.664931778845034 2loss/SRqqt4/assignment_loss‚2.4461795588334403 /loss/FBt1/assignment_loss‚0.408448718703137 /loss/FRt3/detection_loss‚2.5741841752588015 .loss/FRt2/detection_loss‚1.989563441668686 /loss/FRt1/assignment_loss‚3.662590914165851 2loss/SRqqt1/assignment_loss‚1.8802202977575697LÈAÖ ‚(:’ ÓÕâÔŸªßu gpu.0.gpu‚27.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚158.264' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.652800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“ö‡× ‚(:’ ÓÕâÔ¦ÚŒy$ memory_percent‚85.12630462646484 cpu‚23.822842051878123 proc.cpu.threads‚17 disk.sda.in‚0 disk.sda1.out‚0& proc.memory.availableMB‚ 2436.90625 network.sent‚ 1.538224e+06 disk.rbd3.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0% disk./.usageGB‚1042.6204566955566$ disk.nvme0n1.out‚4189.1435546875 disk.nvme0c0n1.in‚44.4375 disk.dm-1.out‚ 34.4296875 disk.sda.out‚ 34.4296875 disk.rbd0.in‚0& disk.nvme0c0n1.out‚4189.1435546875 disk.rbd3.out‚ 1.36328125 disk.sda3.in‚0 disk.rbd1.in‚0 disk.sda2.in‚0* proc.memory.percent‚17.714834213256836! disk.dm-0.out‚4189.1435546875 disk.sda3.out‚ 34.4296875 disk.dm-1.in‚0 disk.rbd2.out‚42.14111328125 disk.rbd0.out‚0.078125" proc.memory.rssMB‚ 2902.3984375 network.recv‚348981 disk.rbd1.out‚ 0.517578125 disk.drbd1777.out‚0( disk./.usagePercent‚69.9711336656707 disk.dm-0.in‚44.4375 disk.rbd2.in‚0 disk.nvme0n1.in‚44.4375 disk.sda1.in‚0æ®`x’Ø ‚(:‡ âÕâÔǤÑu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.756' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9512 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æ·¹“ Ù ‚(:• âÕâÔà—‰x disk.sda3.in‚0 disk.sda.out‚ 35.0546875( proc.memory.availableMB‚ 2436.9765625# proc.memory.rssMB‚ 2902.40234375 proc.cpu.threads‚17 disk.rbd0.out‚0.078125 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚44.4375 disk.sda3.out‚ 35.0546875 disk.rbd1.in‚0 disk.sda1.out‚0$ memory_percent‚85.12587547302246) disk./.usagePercent‚69.97113724977618 disk.drbd1777.out‚0! disk.dm-0.out‚4190.2392578125 disk.rbd2.in‚0 disk.sda.in‚0* proc.memory.percent‚17.714858055114746 cpu‚23.850765384167694 disk.dm-1.in‚0 disk.rbd2.out‚ 43.3056640625$ disk.nvme0n1.out‚4190.2392578125 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚44.4375 disk.sda2.in‚0 disk.rbd1.out‚ 0.517578125 disk.nvme0n1.in‚44.4375 network.recv‚354211 disk.rbd3.in‚0 disk.rbd0.in‚0% disk./.usageGB‚1042.6205101013184 disk.sda2.out‚0 disk.dm-1.out‚ 35.0546875 network.sent‚ 1.624366e+06& disk.nvme0c0n1.out‚4190.2392578125±FÑH’Ú ‚(:‡ ñÕâÔˆ u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.487' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.8974 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ý„« "Û ‚(:— ñÕâÔ¥è®y disk.rbd2.out‚ 44.4111328125$ disk.nvme0n1.out‚4288.5673828125 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 cpu‚23.877978838319013 proc.cpu.threads‚17 disk.dm-1.in‚0! disk.dm-0.out‚4288.5673828125 disk.sda3.out‚ 35.56640625 network.sent‚ 1.629792e+06) disk./.usagePercent‚69.97113724977618 disk.dm-1.out‚ 35.56640625 disk.sda.out‚ 35.56640625 disk.sda1.in‚0 disk.sda3.in‚0 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0% disk./.usageGB‚1042.6205101013184 disk.nvme0c0n1.in‚44.4375 disk.sda2.out‚0 disk.nvme0n1.in‚44.4375& disk.nvme0c0n1.out‚4288.5673828125 disk.sda2.in‚0$ memory_percent‚85.12287139892578& proc.memory.availableMB‚ 2437.46875* proc.memory.percent‚17.714858055114746 network.recv‚357737 disk.sda.in‚0 disk.sda1.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.5498046875 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚44.4375# proc.memory.rssMB‚ 2902.40234375pÔ0Ü + _wandbruntime‚892  _runtime‚892û¬­ÂÝ ¼ _runtime‚ 892.89342322 _step‚77 & lr-AdamW/pg1‚0.0016962989503630658 & lr-AdamW/pg2‚0.0016962989503630658 trainer/global_step‚999 $ _timestamp‚1.788390137137456e+09vp´ÆÞ À trainer/global_step‚999 $ _timestamp‚1.788390137137456e+09 _runtime‚ 892.89342322 _step‚77 & lr-AdamW/pg1‚0.0016962989503630658 & lr-AdamW/pg2‚0.0016962989503630658Mßo0ß + _wandbruntime‚892  _runtime‚892¶š<¤Ç'à ÁO KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15630459 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 810736438627 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚66495775551488 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5290847 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 850012598272 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 815308657664 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16923968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _runtime‚ 892.898041572 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚473 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 815308657664 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 850250530816 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚64930132998144 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16923968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15630935 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚67311101248512 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21177844 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21200440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38101810 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚67310888031232 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 815112479744 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10339612 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 815112479744 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚66495792590848 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚476 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚65740677440239 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21177844 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚66495775551488 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚67868341051904 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0ÚtCË'á ÅO @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 815308657664 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 850250530816 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚64930132998144 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16923968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15630935 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚67311101248512 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21177844 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21200440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38101810 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚67310888031232 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 815112479744 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10339612 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 815112479744 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚66495792590848 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚476 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚65740677440239 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21177844 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚66495775551488 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚67868341051904 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15630459 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 810736438627 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚66495775551488 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5290847 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 850012598272 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 815308657664 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16923968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _runtime‚ 892.898041572 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚473 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0N4/ÐF0â + _wandbruntime‚893  _runtime‚893··³,Ÿã ÊM YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚64993732382720 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚67376725458944 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚16940486 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10349649 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10349646 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16940488 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38161826 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚67934431844864 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚67083591113728 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38139202 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚66560814588416 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5296295 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15645944 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5295782 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚64993749422080 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 811338325043 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21198716 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚999 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38161826 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15645428 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚67083616017408 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 _step‚79 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271211520 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚65805279743863 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788390137897542e+09 _runtime‚ 893.654050451 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚2260n1 22 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚66560814588416 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚67934703056384 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21221338 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246307840 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚65805070707763 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 850840731136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21221338 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚67376938689536 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚67376938689536 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788390137897542e+09 _runtime‚ 893.654050451 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚66560814588416 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚67934703056384 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21221338 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246307840 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚65805070707763 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 850840731136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21221338 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚67376938689536 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚67376938689536 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38161826 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚67934431844864 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚67083591113728 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38139202 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚66560814588416 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5296295 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27326 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15645944 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5295782 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚64993749422080 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 811338325043 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21198716 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚999 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38161826 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15645428 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚67083616017408 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 _step‚79 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545ON΃0å + _wandbruntime‚893  _runtime‚893×5îŠTæ Î .loss/FRt4/detection_loss‚2.416998801024064 0loss/SRqqt4/detection_loss‚2.557551647722721 .loss/FBt1/detection_loss‚3.929606477419535 /loss/FBt2/detection_loss‚0.7459930746312837 2loss/SRqqt3/assignment_loss‚1.9977545313929257 1loss/SRqqt3/detection_loss‚1.9923281945680318 epoch‚1 /loss/FRt4/assignment_loss‚5.126208474670631 0loss/FBt4/assignment_loss‚0.5242894335972563 2loss/SRqqt2/assignment_loss‚1.6171291863136257 1loss/SRqqt1/assignment_loss‚2.073062872293816 .loss/FRt2/detection_loss‚1.720109716475567 1loss/SRqqt1/detection_loss‚1.9528806712678684 /loss/FBt4/detection_loss‚3.0451675465232446 0loss/FRt1/assignment_loss‚3.8424005868534246 1loss/FBt2/assignment_loss‚0.23775279497651994 &loss/total_loss‚52.162490477092916 0loss/FRt3/assignment_loss‚3.8705685093998907 2loss/SRqqt4/assignment_loss‚2.4645994772513706 1loss/FBt1/assignment_loss‚0.17020378490431337 0loss/FBt3/assignment_loss‚0.7758306368237494 /loss/FBt3/detection_loss‚2.2257822709424153 trainer/global_step‚999 _step‚80 .loss/FRt3/detection_loss‚2.561433429270983 0loss/SRqqt2/detection_loss‚1.142376793667357 $ _timestamp‚1.788390137899311e+09 _runtime‚ 893.655151185 /loss/FRt2/assignment_loss‚2.735582545538929 .loss/FRt1/detection_loss‚2.436879019563397~šÞXç Ò 0loss/FRt1/assignment_loss‚3.8424005868534246 1loss/FBt2/assignment_loss‚0.23775279497651994 &loss/total_loss‚52.162490477092916 0loss/FRt3/assignment_loss‚3.8705685093998907 2loss/SRqqt4/assignment_loss‚2.4645994772513706 1loss/FBt1/assignment_loss‚0.17020378490431337 0loss/FBt3/assignment_loss‚0.7758306368237494 /loss/FBt3/detection_loss‚2.2257822709424153 trainer/global_step‚999 _step‚80 .loss/FRt3/detection_loss‚2.561433429270983 0loss/SRqqt2/detection_loss‚1.142376793667357 $ _timestamp‚1.788390137899311e+09 _runtime‚ 893.655151185 /loss/FRt2/assignment_loss‚2.735582545538929 .loss/FRt1/detection_loss‚2.436879019563397 .loss/FRt4/detection_loss‚2.416998801024064 0loss/SRqqt4/detection_loss‚2.557551647722721 .loss/FBt1/detection_loss‚3.929606477419535 /loss/FBt2/detection_loss‚0.7459930746312837 2loss/SRqqt3/assignment_loss‚1.9977545313929257 1loss/SRqqt3/detection_loss‚1.9923281945680318 epoch‚1 /loss/FRt4/assignment_loss‚5.126208474670631 0loss/FBt4/assignment_loss‚0.5242894335972563 2loss/SRqqt2/assignment_loss‚1.6171291863136257 1loss/SRqqt1/assignment_loss‚2.073062872293816 .loss/FRt2/detection_loss‚1.720109716475567 1loss/SRqqt1/detection_loss‚1.9528806712678684 /loss/FBt4/detection_loss‚3.0451675465232446P¿ÑžI‘è ‚(:† €ÖâÔÞöØu gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚96.132' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.2264 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢V·Ó"é ‚(:— €ÖâÔ€²Õx disk.drbd1777.in‚0 disk.rbd2.out‚45.57568359375 disk.nvme0n1.in‚44.4375 disk.sda1.out‚0# proc.memory.rssMB‚ 2902.40234375 disk.sda.in‚0% disk./.usageGB‚1042.6205101013184 disk.sda2.in‚0 disk.dm-0.in‚44.4375 disk.rbd3.out‚ 1.36328125 disk.rbd2.in‚0 disk.sda1.in‚0 disk.rbd0.in‚0 disk.dm-0.out‚4387.400390625 disk.rbd1.in‚0$ memory_percent‚85.12575626373291* proc.memory.percent‚17.714858055114746 disk.drbd1777.out‚0 disk.nvme0c0n1.in‚44.4375% disk.nvme0c0n1.out‚4387.400390625 disk.sda2.out‚0 disk.rbd3.in‚0) proc.memory.availableMB‚ 2436.99609375 network.recv‚362250 disk.sda3.in‚0 disk.dm-1.out‚ 36.07421875 disk.rbd1.out‚ 0.5498046875) disk./.usagePercent‚69.97113724977618 disk.rbd0.out‚0.078125# disk.nvme0n1.out‚4387.400390625 disk.sda3.out‚ 36.07421875 disk.dm-1.in‚0 disk.sda.out‚ 36.07421875 cpu‚23.90472618549617 proc.cpu.threads‚17 network.sent‚ 1.635863e+06õ¥!®œê ‚(:‘ ÖâÔ¹–Íu gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.44' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.287999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0½¢I¼$ë ‚(:™ ÖâÔÚÒÚw disk.drbd1777.out‚0% disk.nvme0c0n1.out‚4387.748046875 disk.sda2.in‚0 disk.sda2.out‚0 cpu‚23.930454530228047 disk.sda3.out‚ 36.57421875 proc.cpu.threads‚17 network.sent‚ 1.723338e+06 disk.nvme0n1.in‚44.4375 disk.sda.in‚0 disk.rbd0.in‚0 disk.nvme0c0n1.in‚44.4375) proc.memory.availableMB‚ 2436.53515625 disk.dm-1.in‚0 disk.dm-1.out‚ 36.57421875# disk.nvme0n1.out‚4387.748046875 disk.sda.out‚ 36.57421875 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚44.4375 disk.rbd1.in‚0 network.recv‚368572) disk./.usagePercent‚69.97113724977618 disk.rbd2.in‚0$ memory_percent‚85.12856960296631# proc.memory.rssMB‚ 2902.40234375 disk.rbd2.out‚46.68505859375 disk.rbd0.out‚0.078125 disk.rbd1.out‚ 0.61279296875% disk./.usageGB‚1042.6205101013184 disk.sda1.out‚0 disk.rbd3.in‚0 disk.dm-0.out‚4387.748046875* proc.memory.percent‚17.714858055114746 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda3.in‚0¼YN7ì ‚(:’ žÖâÔ›Þ³u gpu.0.gpu‚22.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚127.618' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.523599999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0mt/í ‚(:Ž žÖâÔ¶±½w% disk./.usageGB‚1042.6205101013184 disk.rbd1.out‚ 0.61279296875 disk.sda.in‚0$ disk.nvme0c0n1.out‚ 4486.56640625 disk.rbd3.in‚0 disk.dm-0.out‚ 4486.56640625 disk.dm-1.in‚0 disk.rbd2.out‚ 47.849609375 disk.sda2.in‚0) proc.memory.availableMB‚ 2436.78515625 proc.cpu.threads‚17$ memory_percent‚85.12704372406006" disk.nvme0n1.out‚ 4486.56640625 disk.sda3.in‚0* proc.memory.percent‚17.714858055114746 cpu‚23.955643144726853 disk.sda1.in‚0 disk.sda1.out‚0 disk.nvme0c0n1.in‚44.4375 network.recv‚372324 disk.rbd2.in‚0 disk.nvme0n1.in‚44.4375 disk.rbd0.in‚0 disk.drbd1777.in‚0) disk./.usagePercent‚69.97113724977618 disk.sda.out‚ 37.203125 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.dm-0.in‚44.4375# proc.memory.rssMB‚ 2902.40234375 network.sent‚ 1.728516e+06 disk.dm-1.out‚ 37.203125 disk.rbd1.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚ 37.203125p\2‚0î + _wandbruntime‚935  _runtime‚935žs>Åï ¿ trainer/global_step‚1049 % _timestamp‚1.7883901792868562e+09 _runtime‚ 935.042871499 _step‚81 & lr-AdamW/pg1‚0.0016903311434412266 & lr-AdamW/pg2‚0.0016903311434412266N®–Éð à & lr-AdamW/pg1‚0.0016903311434412266 & lr-AdamW/pg2‚0.0016903311434412266 trainer/global_step‚1049 % _timestamp‚1.7883901792868562e+09 _runtime‚ 935.042871499 _step‚81Qµ”#›0ñ + _wandbruntime‚935  _runtime‚935š3õdÆ'ò ÀO KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚70602787245056 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5548565 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10841312 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚39974762 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚69747727397888 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚68960577024339 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚ 935.04425757 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 855059847168 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 850265761503 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16390400 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16389877 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10841315 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚71186404412416 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚17749966 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚69747744437248 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚39997360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚68110119266304 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚70294340433408 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _step‚82 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚69747744437248 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚22224796 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 850457758035 trainer/global_step‚1049 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚71186663054336 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚70294365337088 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233738240 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚68960367988447 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 855059847168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 855256025088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22224796 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258641920 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚68110102226944 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚69747727397888 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17749968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22247392 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚70603000462336 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚520 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 892063979008 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 $ _timestamp‚1.788390179287995e+09 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233738240 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚68960367988447 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 855059847168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 855256025088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22224796 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258641920 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚68110102226944 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚69747727397888 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17749968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22247392 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚70603000462336 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚520 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 892063979008 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 $ _timestamp‚1.788390179287995e+09 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚70602787245056 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5548565 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10841312 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚39974762 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚69747727397888 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚68960577024339 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚ 935.04425757 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 855059847168 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 850265761503 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0R´ÏŒ0ô + _wandbruntime‚935  _runtime‚935gæóÖÒ&õ ÌM ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 851059644451 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10851346 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚69024970292071 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚40034778 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 856054429184 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40034778 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 855858237952 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5553679 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17766488 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22245668 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚70668624672768 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚71253026598912 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚69024761255971 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚22268290 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚83 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚71252778456064 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 856054429184 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚70668837903360 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚1049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17766486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚70668837903360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚69812783474176 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 851251641191 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚69812766434816 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40012154 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16405025 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5554178 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚70359890833408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 248142848 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚40034778 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 856054429184 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40034778 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 855858237952 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5553679 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17766488 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22245668 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚70668624672768 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚71253026598912 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚69024761255971 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚22268290 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚83 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚71252778456064 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 856054429184 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚70668837903360 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚1049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17766486 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚70668837903360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚69812783474176 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 851251641191 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚69812766434816 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40012154 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16405025 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5554178 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚70359890833408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 248142848 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OÊÉ‚ à & lr-AdamW/pg2‚0.0016836694792490298 trainer/global_step‚1099 % _timestamp‚1.7883902213160033e+09 _runtime‚ 977.071978494 _step‚85 & lr-AdamW/pg1‚0.0016836694792490298UÛa0ƒ + _wandbruntime‚977  _runtime‚977 >_ãÈ'„ ÂO M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 934022867456 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23271748 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 934269188608 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73570615249408 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚72180267572547 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 895007214592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5806547 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚72999696283648 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚72180058536655 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18575966 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883902213170378e+09 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11343012 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚72999679244288 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚41847714 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚73894899676160 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 895203392512 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚73894686458880 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 895203392512 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17150057 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚23271748 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚73570640153088 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚72999679244288 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18575966 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5807042 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚1099 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚72999696283648 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚41870312 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚71290088495104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824žP'Ì'… ÆO TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚73894686458880 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚74504909341696 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚73894686458880 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 895203392512 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17150057 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚23271748 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217280 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚73570640153088 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚72999679244288 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18575966 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5807042 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚1099 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚72999696283648 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚41870312 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚71290088495104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 934022867456 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23271748 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 934269188608 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73570615249408 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚72180267572547 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035892 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 895007214592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5806547 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚72999696283648 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚72180058536655 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18575966 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883902213170378e+09 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11343012 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚72999679244288 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚41847714 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚73894899676160 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 895203392512 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 IDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚1099 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254696448 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚73960737117184 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚517 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚74571267215360 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41907730 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17164682 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 890972960599 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18592488 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23292620 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 895805605376 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚ 977.830423274 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11353049 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11353046 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23315242 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 896001796608 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 890780963859 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _step‚87 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚41907730 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 E?Devië¶[%ceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚73636140745728 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚73960737117184ò…óúÖ&ˆ ÐM WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚73064718281216 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 895805605376 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18592486 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17164682 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 890972960599 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18592488 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23292620 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 895805605376 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚ 977.830423274 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11353049 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11353046 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23315242 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 896001796608 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 890780963859 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _step‚87 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚41907730 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚73636140745728 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚73960737117184 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322689536 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18592488 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚514 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚71353670840320 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚73960523886592 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚73064718281216 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚72244660840279 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚72244451804179 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5812150 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚1099 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213230592 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254696448 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚73960737117184 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚517 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚74571267215360 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41907730 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 976276722688 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 935150759936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43720666 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17909950 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚526 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126848 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 929708400319 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚24318700 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 934954582016 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚76251648130048 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚76251648130048 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17909421 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚76846890065408 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 934954582016 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚77822924661248 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚77186585672704 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚76846914969088 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 935150759936 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24341296 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚75399958120755 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _runtime‚1018.846117835 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19401968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19401966 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883902630895889e+09 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43743264 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 929900396851 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚77186798889984 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚529 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 976276722688 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 935150759936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196177920 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22598 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43720666 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17909950 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚526 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126848 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498533888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322689536 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 929708400319 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚24318700 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 934954582016 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚76251648130048 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚76251648130048 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17909421 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚76846890065408 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 934954582016 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚77822924661248 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚77186585672704 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322689536 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚76846914969088 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 935150759936 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24341296 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocat}'ª`ed‚75399958120755 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _runtime‚1018.846117835 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19401968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19401966 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883902630895889e+09 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43743264 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 929900396851 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚77186798889984 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚529 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 935752972800 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43780682 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17925156 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚76316687166976 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6070407 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚24339572 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883902638360353e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11854746 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚1019.592601683 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17924654 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6069908 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚76316687166976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75464142352387 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43758058 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚76912440465408 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚76316670127616 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24362194 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11854749 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75464351388487 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19418486 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19418486 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 935752972800 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 935949164032 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚74533657108480 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚77252423100416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19418488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚499 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚77889294033408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚76316670127616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚77889554759168 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19418488 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚77252636331008 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888[q¥j×&˜ ÑM L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚76316670127616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚77889554759168 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498533888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19418488 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚77252636331008 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43758058 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24339572 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 976878471680 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 935752972800 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43780682 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17925156 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚76316687166976 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6070407 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27579 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚24339572 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191232 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883902638360353e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11854746 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚1019.592601683 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17924654 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6069908 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚76316687166976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75464142352387 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43758058 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚76912440465408 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚76316670127616 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24362194 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11854749 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75464351388487 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19418486 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19418486 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 935752972800 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 935949164032 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚74533657108480 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚77252423100416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19418488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚499 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚77889294033408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 FÀ„2¥ - _wandbruntime‚1060  _runtime‚1060\£ ¦ ÅO VEDe4¢ô¿'viceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 974901949440 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚94 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25388248 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚80123164881408 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚78619648668963 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚1060.906983459 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80478484886528 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚79503599976448 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20227966 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 974901949440 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚77650009913344 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 975098127360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚81141112142336 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚78619439633071 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25365652 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚1199 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80123189785088 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235835392 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883903051475737e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20227968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚502 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12346415 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1018183096320 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6324002 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1017947260928 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12346412 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6324501 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 969621716259 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚499 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚77650026952704 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45593618 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80478484886528 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507j©Ï'§ ÉO IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27326 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 975098127360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22596 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚81141112142336 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚78619439633071 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25365652 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22598 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚1199 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27579 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80123189785088 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235835392 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883903051475737e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27579 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20227968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚502 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12346415 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1018183096320 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996532 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6324002 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1017947260928 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12346412 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6324501 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 969621716259 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345545 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22596 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498533888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚499 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚77650026952704 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45593618 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217280 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80478484886528 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196177920 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 974901949440 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚94 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25388248 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚80123164881408 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚78619648668963 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚1060.906983459 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80478484886528 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚79503599976448 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27326 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640705 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20227966 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 974901949440 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚77650009913344 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0^ÕÁìè2¨ - _wandbruntime‚1061  _runtime‚1061ùZqšÕ&© ÏM D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883903058963838e+09 _runtime‚1061.652983925 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80544322314240 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20244486 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20244486 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252599296 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20244488 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚45653634 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18686172 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 975700340224 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 975700340224 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18685667 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12356449 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚79568621974016 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45653634 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45631010 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25409146 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 970415599415 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80544535544832 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1018773460480 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277502976 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25386524 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 970223602675 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚79568639013376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚79568621974016 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚77713609297920 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 975896531456 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚81207463838208 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚80188715281408 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77713626337280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚80544535544832 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _step‚95 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚79568639013376 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25409146 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚505 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12356446 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚81207741341184 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚78683832900595 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚45631010 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20244488 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚80188690377728 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0ÛÔÈýH ª ÓM SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345545 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277502976 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22622 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25386524 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 970223602675 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚79568639013376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚79568621974016 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚77713609297920 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 975896531456 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚81207463838208 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚80188715281408 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996740 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213230592 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77713626337280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚80544535544832 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _step‚95 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚79568639013376 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25409146 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚505 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12356446 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚81207741341184 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚78683832900595 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚45631010 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20244488 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚80188690377728 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22622 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883903058963838e+09 _runtime‚1061.652983925 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80544322314240 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20244486 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20244486 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191232 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252599296 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20244488 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚45653634 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18686172 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322689536 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22624 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 975700340224 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 975700340224 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18685667 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12356449 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚79568621974016 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45653634 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27579 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640705 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45631010 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25409146 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 970415599415 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80544535544832 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498533888 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1018773460480 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036100_¡¶u¢2« - _wandbruntime‚1061  _runtime‚1061‘Ý-0X¬ Ò /loss/FBt1/detection_loss‚2.5019345097243786 /loss/FRt1/assignment_loss‚3.315767502272001 1loss/SRqqt4/detection_loss‚2.6068202647669563 trainer/global_step‚1199 /loss/FRt4/assignment_loss‚4.188204342668707 0loss/FBt4/assignment_loss‚0.5667395797354402 /loss/FRt3/detection_loss‚2.7648610112382404 &loss/total_loss‚47.898639284260575 _step‚96 2loss/SRqqt2/assignment_loss‚1.5537384952638895 /loss/FRt1/detection_loss‚2.2388007493429285 1loss/SRqqt2/detection_loss‚1.1164272243853743 -loss/FBt2/detection_loss‚0.64982772531763 /loss/FBt3/detection_loss‚2.7787517490131513 % _timestamp‚1.7883903058983474e+09 2loss/SRqqt1/assignment_loss‚1.7346705874571433 2loss/SRqqt4/assignment_loss‚2.4056459989013343 1loss/SRqqt3/detection_loss‚1.9997513220551308 .loss/FBt4/detection_loss‚2.766339804444994 epoch‚1 _runtime‚1061.654131177 1loss/SRqqt3/assignment_loss‚2.096038759591874 1loss/FBt2/assignment_loss‚0.13022021749537202 0loss/FBt3/assignment_loss‚0.5617849162302134 /loss/FRt2/detection_loss‚1.7276706658303738 .loss/FRt4/detection_loss‚2.808516496961767 0loss/SRqqt1/detection_loss‚1.678538471715063 1loss/FBt1/assignment_loss‚0.18774895815431591 0loss/FRt2/assignment_loss‚2.0783618287398267 /loss/FRt3/assignment_loss‚3.441478102954466‡­~ \­ Ö 0loss/FRt2/assignment_loss‚2.0783618287398267 /loss/FRt3/assignment_loss‚3.441478102954466 /loss/FBt1/detection_loss‚2.5019345097243786 /loss/FRt1/assignment_loss‚3.315767502272001 1loss/SRqqt4/detection_loss‚2.6068202647669563 trainer/global_step‚1199 /loss/FRt4/assignment_loss‚4.188204342668707 0loss/FBt4/assignment_loss‚0.5667395797354402 /loss/FRt3/detection_loss‚2.7648610112382404 &loss/total_loss‚47.898639284260575 _step‚96 2loss/SRqqt2/assignment_loss‚1.5537384952638895 /loss/FRt1/detection_loss‚2.2388007493429285 1loss/SRqqt2/detection_loss‚1.1164272243853743 -loss/FBt2/detection_loss‚0.64982772531763 /loss/FBt3/detection_loss‚2.7787517490131513 % _timestamp‚1.7883903058983474e+09 2loss/SRqqt1/assignment_loss‚1.7346705874571433 2loss/SRqqt4/assignment_loss‚2.4056459989013343 1loss/SRqqt3/detection_loss‚1.9997513220551308 .loss/FBt4/detection_loss‚2.766339804444994 epoch‚1 _runtime‚1061.654131177 1loss/SRqqt3/assignment_loss‚2.096038759591874 1loss/FBt2/assignment_loss‚0.13022021749537202 0loss/FBt3/assignment_loss‚0.5617849162302134 /loss/FRt2/detection_loss‚1.7276706658303738 .loss/FRt4/detection_loss‚2.808516496961767 0loss/SRqqt1/detection_loss‚1.678538471715063 1loss/FBt1/assignment_loss‚0.18774895815431591`\V¼ú® ‚(:… ¥×âÔ­ª­u gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.405' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.881 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0šVwC¯ ‚(:¸ ¥×âÔüð…x% disk./.usageGB‚1042.6185417175293 disk.rbd2.out‚58.56103515625! disk.nvme0c0n1.in‚ 48.39453125 proc.cpu.threads‚17 network.recv‚414982 disk.sda2.out‚0 disk.rbd3.in‚0 disk.rbd0.in‚0 network.sent‚ 2.023004e+06 disk.sda3.out‚ 42.52734375) disk./.usagePercent‚69.97100514988853 disk.sda3.in‚0.109375$ disk.nvme0n1.out‚5085.7314453125 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ memory_percent‚85.12740135192871( proc.memory.availableMB‚ 2436.7265625 disk.dm-0.in‚ 48.39453125 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.rbd0.out‚0.078125* proc.memory.percent‚17.714858055114746 disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.out‚ 0.67578125 disk.rbd2.in‚0 disk.nvme0n1.in‚ 48.39453125 disk.sda.in‚0.109375 disk.sda1.in‚0# proc.memory.rssMB‚ 2902.40234375 disk.dm-1.out‚ 42.52734375 disk.sda.out‚ 42.52734375 disk.sda1.out‚0! disk.dm-0.out‚5085.7314453125& disk.nvme0c0n1.out‚5085.7314453125 cpu‚24.15143888509981s]Ý° ‚(:… ´×âÔáÓÒu gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.876' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7752 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¡–´/@± ‚(:µ ´×âÔ´®åw disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.in‚0.109375 disk.sda.out‚ 42.8828125) proc.memory.availableMB‚ 2434.89453125) disk./.usagePercent‚69.97100002973784 disk.sda1.out‚0* proc.memory.percent‚17.725276947021484 disk.sda3.out‚ 42.8828125% disk.nvme0n1.out‚5087.47119140625 disk.drbd1777.in‚0! proc.memory.rssMB‚ 2904.109375 disk.dm-0.in‚ 49.11328125" disk.dm-0.out‚5087.47119140625 disk.dm-1.out‚ 42.8828125 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 network.sent‚ 2.11016e+06$ disk./.usageGB‚1042.618465423584' disk.nvme0c0n1.out‚5087.47119140625 disk.rbd1.out‚ 0.67578125 disk.rbd2.in‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.rbd3.in‚0 disk.rbd2.out‚60.76416015625 disk.nvme0n1.in‚ 49.11328125 disk.sda1.in‚0$ memory_percent‚85.13858318328857 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 proc.cpu.threads‚17 network.recv‚420994! disk.nvme0c0n1.in‚ 49.11328125 cpu‚24.23945191813054¤ß:š² ‚(: Ã×âԻʤu gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚30.0 gpu.0.powerWatts‚88.58' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.715999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0íbnú>³ ‚(:³ Ã×âÔ¨íØy disk.dm-1.out‚ 43.515625 network.recv‚425741 disk.rbd2.in‚0% disk.nvme0n1.out‚5185.82275390625 disk.sda.in‚0.109375 disk.sda3.in‚0.109375) disk./.usagePercent‚69.97098953342893 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.sda.out‚ 43.515625 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 49.11328125 disk.rbd1.in‚0! proc.memory.rssMB‚ 2904.109375 disk.rbd2.out‚61.77587890625 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0" disk.dm-0.out‚5185.82275390625( proc.memory.availableMB‚ 2434.4921875 disk.nvme0n1.in‚ 49.11328125 disk.rbd1.out‚ 0.67578125 cpu‚24.334344457430173 network.sent‚ 2.116759e+06$ memory_percent‚85.14103889465332 proc.cpu.threads‚17$ disk./.usageGB‚1042.618309020996! disk.nvme0c0n1.in‚ 49.11328125' disk.nvme0c0n1.out‚5185.82275390625* proc.memory.percent‚17.725276947021484 disk.sda3.out‚ 43.515625 disk.dm-1.in‚0.109375Þ?ª¶2´ - _wandbruntime‚1108  _runtime‚11087Çi°ˆEµ æ¦ particle/accuracy_1_of_1‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9906522035598755 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 jet/accuracy_7_of_7‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1428571492433548 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3224773108959198 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3205822706222534 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_1_of_5‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 epoch‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16834454238414764 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2351255714893341 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ;"Purity/1FRt1SRqqt2FBt/event_purity‚0.011904762126505375 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 6 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.281110554933548 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.40871113538742065 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4245748221874237 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.09756948053836823 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 +jet/accuracy_3_of_6‚0.07894736528396606 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.41341522336006165 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.47000259160995483 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.30334097146987915 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.13739974796772003 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.07951368391513824 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 particle/accuracy_3_of_3‚1 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.043251752853393555 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3190504014492035 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.0854426920413971 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12014798074960709 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6712651252746582 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25605958700180054 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.18663790822029114 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9985975027084351 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3239441514015198 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3053734600543976 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38953617215156555 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 trainer/global_step‚1207 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.261904776096344 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.13873979449272156 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03644800931215286 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2963179051876068 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1651270091533661 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3583333194255829 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3302256762981415 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3325065076351166 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.11999999731779099 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3422639071941376 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.31125470995903015 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.18686868250370026 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.34192973375320435 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17614975571632385 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.06790123879909515 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.034485407173633575 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.49444445967674255 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.15740740299224854 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013963291421532631 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05859598517417908 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.365168958902359 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3223806321620941 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.012544803321361542 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.32098767161369324 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.07036449760198593 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2557254433631897 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 'particle/f_score‚0.3708389699459076 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.0872797816991806 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.37276822328567505 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.3880555033683777 9"Purity/1FRt0SRqqt2FBt/event_purity‚0.0637301579117775 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.35663414001464844 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.2528060972690582 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4020850658416748 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36336061358451843 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.30035167932510376 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3300085663795471 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 % _timestamp‚1.7883903525624673e+09 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1428571492433548 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.021903367713093758 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.367499977350235 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.43942171335220337 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4459165334701538 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08207295835018158 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.013267338275909424 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1574462503194809 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.02500000037252903 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2953515648841858 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_4_of_6‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.32894739508628845 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9859517812728882 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.002600369043648243 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.699999988079071 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14080087840557098 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5701754689216614 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9891338348388672 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995580911636353 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0933597981929779 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9156746864318848 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.31600528955459595 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9985803961753845 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.14604921638965607 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.35897430777549744 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.23076923191547394 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.0998413935303688 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8930041790008545 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03329508751630783 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3295042812824249 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3214285671710968 particle/accuracy_1_of_6‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9892921447753906 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.03420634940266609 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.28338804841041565 *jet/accuracy_2_of_5‚0.4254443049430847 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.31264281272888184 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9889031052589417 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 *jet/accuracy_1_of_1‚0.3995085060596466 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 )loss/val_total_loss‚50.46502639481665 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.3849734961986542 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3093104064464569 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03514939174056053 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4065476059913635 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3397429883480072 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9887498021125793 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 particle/accuracy_4_of_5‚1 particle/accuracy_1_of_7‚1 8"Purity/*FRt0SRqqt3FBt/event_purity‚0.065476194024086 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_4‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.06833618879318237 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3106575906276703 5 Purity/*FRt3S\VWåMRqqt0FBt/FRt_purity‚0.14823018014431 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6318712830543518 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6052631735801697 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8146815896034241 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4044877886772156 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.30964159965515137 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3597096800804138 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.08794150501489639 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005461633671075106 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.14084742963314056 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.0748792290687561 particle/accuracy_2_of_5‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.1185101866722107 *jet/accuracy_2_of_4‚0.2899220883846283 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.09335310012102127 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10519131273031235 )jet/accuracy_1_of_3‚0.756782054901123 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.11406926810741425 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4077834188938141 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.15307220816612244 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3069131374359131 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9731481075286865 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4444267749786377 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.01579725369811058 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9792627692222595 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9766837358474731 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333730697632 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.35555556416511536 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0065977000631392 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4404761791229248 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9868044257164001 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.04091658443212509 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.3499999940395355 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 _step‚97 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3891558349132538 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3433654010295868 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8251776695251465 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 jet/accuracy_2_of_7‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16870148479938507 +particle/specificity‚0.9445088505744934 particle/accuracy_2_of_6‚1 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9958208203315735 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 jet/accuracy_5_of_7‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 +jet/accuracy_4_of_5‚0.00913978461176157 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34108203649520874 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9897342324256897 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.08850467950105667 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,jet/accuracy_3_of_3‚0.017836572602391243 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.992020308971405 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +jet/accuracy_2_of_2‚0.09260598570108414 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 +jet/accuracy_2_of_6‚0.48245611786842346 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4099845588207245 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.3499999940395355 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.04885057732462883 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3165619671344757 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35841405391693115 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.08554317057132721 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4279342591762543 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.14455312490463257 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.30115440487861633 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.261904776096344 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9841662645339966 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8970507979393005 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9894930124282837 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.14679454267024994 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.14132104814052582 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0833333358168602 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9155556559562683 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.33552631735801697 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.34375 particle/accuracy_5_of_6‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9070881605148315 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.14111554622650146 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12216562032699585 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39418789744377136 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986159801483154 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.013499988242983818 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_2_of_7‚1 _runtime‚1108.320631722 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9902071952819824 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.40764299035072327 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.14390474557876587 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 particle/accuracy_6_of_6‚1 particle/accuracy_6_of_7‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.992048442363739 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3357718586921692 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2695011496543884 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7validation_average_jet_accuracy‚0.36510714888572693 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.29028570652008057 jet/accuracy_5_of_5‚0 particle/accuracy_2_of_3‚1 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2850334346294403 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 particle/accuracy_3_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0011123470030725002 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2705000042915344 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.4117647111415863 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2739218473434448 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3878648579120636 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.22410713136196136 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08637277036905289 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3440741002559662 ,jet/accuracy_3_of_4‚0.031306639313697815 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08082953095436096 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.35555553436279297 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.30820104479789734 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_3‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19000478088855743 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4052821695804596 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_6‚0.9298246502876282 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4279593229293823 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9897118210792542 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3597096800804138 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 )jet/accuracy_1_of_2‚0.636612594127655 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.07066347450017929 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13159355521202087 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2596937119960785 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3194444477558136 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4161667227745056 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4274739921092987 particle/accuracy_1_of_4‚1 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32696110010147095 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.14083164930343628 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20223182439804077 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9934050440788269 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07899387925863266 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333730697632 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.252525269985199 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3090955913066864 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.07516340166330338 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10277777910232544 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_3‚0.20058606564998627 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08207295835018158 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9821428060531616 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.998068630695343 ,particle/sensitivity‚0.29592621326446533 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.14620406925678253 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3482103645801544 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9980875849723816 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_5_of_7‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 (particle/accuracy‚0.8432093262672424 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9912087321281433 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0ßÜ* 2¶ ê¦ :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32696110010147095 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.14083164930343628 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20223182439804077 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9934050440788269 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07899387925863266 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333730697632 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.252525269985199 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3090955913066864 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.07516340166330338 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10277777910232544 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_3‚0.20058606564998627 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08207295835018158 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9821428060531616 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.998068630695343 ,particle/sensitivity‚0.29592621326446533 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.14620406925678253 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3482103645801544 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9980875849723816 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_5_of_7‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 (particle/accuracy‚0.8432093262672424 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9912087321281433 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9906522035598755 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 jet/accuracy_7_of_7‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1428571492433548 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3224773108959198 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3205822706222534 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_1_of_5‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 epoch‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16834454238414764 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2351255714893341 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ;"Purity/1FRt1SRqqt2FBt/event_purity‚0.011904762126505375 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 6 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.281110554933548 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.40871113538742065 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4245748221874237 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.09756948053836823 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 +jet/accuracy_3_of_6‚0.07894736528396606 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.41341522336006165 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.47000259160995483 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.30334097146987915 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.13739974796772003 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.07951368391513824 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 particle/accuracy_3_of_3‚1 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.043251752853393555 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3190504014492035 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.0854426920413971 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12014798074960709 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6712651252746582 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25605958700180054 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.18663790822029114 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9985975027084351 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3239441514015198 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3053734600543976 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38953617215156555 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 trainer/global_step‚1207 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.261904776096344 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.13873979449272156 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03644800931215286 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2963179051876068 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1651270091533661 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3583333194255829 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3302256762981415 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3325065076351166 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.11999999731779099 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3422639071941376 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.31125470995903015 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.18686868250370026 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.34192973375320435 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17614975571632385 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.06790123879909515 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.034485407173633575 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.49444445967674255 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.15740740299224854 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013963291421532631 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05859598517417908 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.365168958902359 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3223806321620941 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.012544803321361542 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.32098767161369324 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.07036449760198593 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2557254433631897 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 'particle/f_score‚0.3708389699459076 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.0872797816991806 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.37276822328567505 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.3880555033683777 9"Purity/1FRt0SRqqt2FBt/event_purity‚0.0637301579117775 :"Purity/1FRt1SRqqt*FBt/SRqqt_puri¬IÞZdaty‚0.35663414001464844 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.2528060972690582 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4020850658416748 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36336061358451843 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.30035167932510376 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3300085663795471 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 % _timestamp‚1.7883903525624673e+09 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1428571492433548 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.021903367713093758 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.367499977350235 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.43942171335220337 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4459165334701538 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08207295835018158 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.013267338275909424 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1574462503194809 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.02500000037252903 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2953515648841858 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_4_of_6‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.32894739508628845 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9859517812728882 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.002600369043648243 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.699999988079071 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14080087840557098 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5701754689216614 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9891338348388672 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995580911636353 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0933597981929779 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9156746864318848 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.31600528955459595 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9985803961753845 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.14604921638965607 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.35897430777549744 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.23076923191547394 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.0998413935303688 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8930041790008545 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03329508751630783 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3295042812824249 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3214285671710968 particle/accuracy_1_of_6‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9892921447753906 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.03420634940266609 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.28338804841041565 *jet/accuracy_2_of_5‚0.4254443049430847 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.31264281272888184 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9889031052589417 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 *jet/accuracy_1_of_1‚0.3995085060596466 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 )loss/val_total_loss‚50.46502639481665 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.3849734961986542 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3093104064464569 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03514939174056053 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4065476059913635 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3397429883480072 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9887498021125793 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 particle/accuracy_4_of_5‚1 particle/accuracy_1_of_7‚1 8"Purity/*FRt0SRqqt3FBt/event_purity‚0.065476194024086 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_4‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.06833618879318237 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3106575906276703 5 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.14823018014431 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6318712830543518 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6052631735801697 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8146815896034241 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4044877886772156 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.30964159965515137 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3597096800804138 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.08794150501489639 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005461633671075106 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.14084742963314056 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.0748792290687561 particle/accuracy_2_of_5‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.1185101866722107 *jet/accuracy_2_of_4‚0.2899220883846283 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.09335310012102127 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10519131273031235 )jet/accuracy_1_of_3‚0.756782054901123 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.11406926810741425 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4077834188938141 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.15307220816612244 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3069131374359131 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9731481075286865 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4444267749786377 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.01579725369811058 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9792627692222595 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9766837358474731 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333730697632 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.35555556416511536 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0065977000631392 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4404761791229248 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9868044257164001 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.04091658443212509 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.3499999940395355 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 _step‚97 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3891558349132538 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3433654010295868 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8251776695251465 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 jet/accuracy_2_of_7‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16870148479938507 +particle/specificity‚0.9445088505744934 particle/accuracy_2_of_6‚1 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9958208203315735 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 jet/accuracy_5_of_7‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 +jet/accuracy_4_of_5‚0.00913978461176157 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34108203649520874 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9897342324256897 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.08850467950105667 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,jet/accuracy_3_of_3‚0.017836572602391243 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.992020308971405 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +jet/accuracy_2_of_2‚0.09260598570108414 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 +jet/accuracy_2_of_6‚0.48245611786842346 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4099845588207245 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.3499999940395355 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.04885057732462883 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3165619671344757 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35841405391693115 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.08554317057132721 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4279342591762543 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.14455312490463257 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.30115440487861633 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.261904776096344 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9841662645339966 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8970507979393005 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9894930124282837 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.14679454267024994 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.14132104814052582 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0833333358168602 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9155556559562683 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.33552631735801697 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.34375 particle/accuracy_5_of_6‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9070881605148315 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.14111554622650146 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12216562032699585 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39418789744377136 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986159801483154 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.013499988242983818 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_2_of_7‚1 _runtime‚1108.320631722 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9902071952819824 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.40764299035072327 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.14390474557876587 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 particle/accuracy_6_of_6‚1 particle/accuracy_6_of_7‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.992048442363739 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3357718586921692 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2695011496543884 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7validation_average_jet_accuracy‚0.36510714888572693 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.29028570652008057 jet/accuracy_5_of_5‚0 particle/accuracy_2_of_3‚1 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2850334346294403 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 particle/accuracy_3_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0011123470030725002 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2705000042915344 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.4117647111415863 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2739218473434448 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3878648579120636 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.22410713136196136 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08637277036905289 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3440741002559662 ,jet/accuracy_3_of_4‚0.031306639313697815 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08082953095436096 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.35555553436279297 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.30820104479789734 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_3‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19000478088855743 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4052821695804596 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_6‚0.9298246502876282 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4279593229293823 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9897118210792542 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3597096800804138 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 )jet/accuracy_1_of_2‚0.636612594127655 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.07066347450017929 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13159355521202087 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2596937119960785 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3194444477558136 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4161667227745056 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4274739921092987 particle/accuracy_1_of_4‚1 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0alØÌŒ›· ‚(: Ò×âÔ”ŵu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚30.0 gpu.0.powerWatts‚75.956' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚15.191200000000002 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0S NF¸ ‚(:» Ò×âÔ§ŸÉx# proc.memory.rssMB‚ 2910.24609375) proc.memory.percent‚17.76273250579834 cpu‚24.41734264445175 disk.dm-1.out‚ 43.90234375 disk.rbd1.out‚ 0.67578125 disk.nvme0n1.in‚ 49.19921875 disk.sda.out‚ 43.90234375 disk.sda1.in‚0 disk.dm-0.in‚ 49.19921875" disk.dm-0.out‚5284.53564453125 disk.dm-1.in‚0.109375 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd2.out‚62.98681640625! disk.nvme0c0n1.in‚ 49.19921875 proc.cpu.threads‚17 disk.rbd0.out‚0.078125% disk./.usageGB‚1042.7610664367676 disk.sda2.in‚0 disk.sda2.out‚0% disk.nvme0n1.out‚5284.53564453125) proc.memory.availableMB‚ 2285.33984375 network.sent‚ 2.121969e+06' disk.nvme0c0n1.out‚5284.53564453125 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.sda3.in‚0.109375 disk.sda3.out‚ 43.90234375 network.recv‚429381) disk./.usagePercent‚69.98057010338282 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0$ memory_percent‚86.05139255523682l2¼â‘¹ ‚(:† á×âÔó­àu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚110.854' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.1708 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0cwŸÌ4º ‚(:© á×âÔÜÓåw disk.sda1.out‚0! disk.nvme0c0n1.out‚ 5427.96875% disk./.usageGB‚1042.7612762451172 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.dm-0.in‚ 49.19921875 disk.sda3.out‚ 44.71484375 disk.rbd1.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.out‚ 0.67578125$ memory_percent‚86.05248928070068) proc.memory.availableMB‚ 2285.16015625 proc.cpu.threads‚17 network.sent‚ 2.224716e+06 disk.sda.in‚0.109375! disk.nvme0c0n1.in‚ 49.19921875 disk.dm-1.out‚ 44.71484375) proc.memory.percent‚17.76280403137207 disk.rbd2.in‚0 disk.sda.out‚ 44.71484375 disk.sda2.out‚0 cpu‚24.431194478153905 disk.nvme0n1.out‚ 5427.96875 disk.rbd3.in‚0 network.recv‚434965 disk.rbd0.out‚0.078125 disk.sda3.in‚0.109375) disk./.usagePercent‚69.98058418379719 disk.rbd2.out‚63.99853515625 disk.drbd1777.in‚0 disk.dm-0.out‚ 5427.96875 disk.dm-1.in‚0.109375" proc.memory.rssMB‚ 2910.2578125 disk.nvme0n1.in‚ 49.19921875(±"›» ‚(: ð×âÔÄ‹Ãx gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.164' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.832800000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0åRÃ9¼ ‚(:® ð×âÔ½Èáx% disk./.usageGB‚1042.7611694335938 disk.sda.out‚ 45.5546875 disk.sda1.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17 disk.rbd1.out‚ 0.67578125# disk.nvme0n1.out‚5674.189453125 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0 network.recv‚439634) disk./.usagePercent‚69.98057701558623 disk.rbd3.in‚0 disk.rbd0.out‚0.078125 disk.dm-0.out‚5674.189453125" proc.memory.rssMB‚ 2910.2578125 cpu‚24.445645675598804 disk.nvme0c0n1.in‚ 49.2265625 disk.sda2.in‚0 disk.rbd2.in‚0( proc.memory.availableMB‚ 2285.1328125 disk.nvme0n1.in‚ 49.2265625 disk.rbd0.in‚0% disk.nvme0c0n1.out‚5674.189453125 disk.dm-1.in‚0.109375 disk.rbd2.out‚66.19775390625 disk.sda.in‚0.109375 disk.dm-0.in‚ 49.2265625 disk.sda3.in‚0.109375 disk.sda3.out‚ 45.5546875 disk.dm-1.out‚ 45.5546875) proc.memory.percent‚17.76280403137207 network.sent‚ 2.230882e+06 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ memory_percent‚86.05265617370605|Æ`&2½ - _wandbruntime‚1145  _runtime‚1145j?Úøľ ¾ _runtime‚1145.899839159 _step‚98 % lr-AdamW/pg1‚0.001659579831628224 % lr-AdamW/pg2‚0.001659579831628224 trainer/global_step‚1249 % _timestamp‚1.7883903901438744e+09)cnæÈ¿  trainer/global_step‚1249 % _timestamp‚1.7883903901438744e+09 _runtime‚1145.899839159 _step‚98 % lr-AdamW/pg1‚0.001659579831628224 % lr-AdamW/pg2‚0.001659579831628224b+ñ%2À - _wandbruntime‚1145  _runtime‚1145%ÓØÞ Á ËO N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883903901449144e+09 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚556 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1025083096337 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚84187513607168 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚553 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚85942362161664 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _step‚99 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚85218556015104 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19848135 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13146087 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6702051 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚82139838060544 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 TGDeviceStatsMonitoO„ 7ór.on_train_batch_start/allocation.small_pool.allocated‚26681231 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚85218342797312 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21448968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21448968 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13146084 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320640 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚48107602 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1030829190144 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21448970 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚84187513607168 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26658634 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚82139821021184 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507Q<ü‡Õ' ÏO TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚21448970 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚85218342797312 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚26681231 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚85942090937344 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1031025368576 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6702604 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7883903901449144e+09 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚556 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1025083096337 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚84187513607168 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚553 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚85942362161664 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _step‚99 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚85218556015104 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19848135 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13146087 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6702051 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚82139838060544 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26681231 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚85218342797312 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21448968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21448968 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13146084 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320640 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚48107602 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1030829190144 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21448970 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚84187513607168 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26658634 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚82139821021184 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 IDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19863157 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1026068976029 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19863689 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚84931239743488 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚100 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1031627580928 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚84252569683456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26679506 _runtime‚ 1146.64986667 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242113024 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13156121 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚84931214839808 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1077243628544 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 % _timestamp‚1.7883903908933344e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21465488 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚48144994 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1249 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚85284393456128 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6707568 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚85284180225024 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82203420405760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚84252552644096 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21465490 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1077485741568 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚84252569683456 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1031627580928 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26679506 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21465490 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13156118 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48167619 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6707039 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚86008725485056 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚82203437445120 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚85284393456128 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚100 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1031627580928 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚84252569683456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26679506 _runtime‚ 1146.64986667 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242113024 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13156121 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚84931214839808 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1077243628544 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 % _timestamp‚1.7883903908933344e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21465488 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚48144994 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1249 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚85284393456128 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6707568 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚85284180225024 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82203420405760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚84252552644096 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21465490 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1077485741568 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚84252569683456 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1031627580928 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26679506 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21465490 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13156118 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48167619 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6707039 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚86008725485056 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚82203437445120 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚85284393456128 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19863157 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1026068976029 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709d—Ùc†2Æ - _wandbruntime‚1146  _runtime‚1146¬kù¸[Ç Õ /loss/FRt2/detection_loss‚1.7020083910279107 0loss/FRt1/assignment_loss‚3.4887642772574172 1loss/SRqqt3/detection_loss‚1.9983433779849802 /loss/FBt2/detection_loss‚0.6054487008049918 1loss/SRqqt2/detection_loss‚1.0367146478496867 .loss/FBt3/detection_loss‚2.427728669345379 %loss/total_loss‚48.74764562002435 trainer/global_step‚1249 0loss/FBt3/assignment_loss‚0.7628525499254465 1loss/FBt4/assignment_loss‚0.43005309531387564 .loss/FRt4/assignment_loss‚4.58375333867422 0loss/FBt1/assignment_loss‚0.5304068306161623 1loss/SRqqt4/detection_loss‚2.6797009447835527 /loss/FRt4/detection_loss‚2.4491827960421397 1loss/SRqqt1/detection_loss‚2.0068417542641708 /loss/FRt3/assignment_loss‚3.679121147183811 2loss/SRqqt2/assignment_loss‚1.4107899493504343 2loss/SRqqt3/assignment_loss‚2.0049086631305757 /loss/FBt2/assignment_loss‚0.260787174098105 _step‚101 0loss/FRt2/assignment_loss‚1.9917734694062619 _runtime‚1146.650873056 /loss/FBt4/detection_loss‚2.8493446509043374 epoch‚2 % _timestamp‚1.7883903908950865e+09 /loss/FRt1/detection_loss‚2.3972484743908833 2loss/SRqqt1/assignment_loss‚1.8826956051853623 /loss/FBt1/detection_loss‚2.8463953009673526 2loss/SRqqt4/assignment_loss‚1.8932917354241856 /loss/FRt3/detection_loss‚2.8294900760931125/z0O_È Ù /loss/FRt2/detection_loss‚1.7020083910279107 0loss/FRt1/assignment_loss‚3.4887642772574172 1loss/SRqqt3/detection_loss‚1.9983433779849802 /loss/FBt2/detection_loss‚0.6054487008049918 1loss/SRqqt2/detection_loss‚1.0367146478496867 .loss/FBt3/detection_loss‚2.427728669345379 %loss/total_loss‚48.74764562002435 trainer/global_step‚1249 0loss/FBt3/assignment_loss‚0.7628525499254465 1loss/FBt4/assignment_loss‚0.43005309531387564 .loss/FRt4/assignment_loss‚4.58375333867422 0loss/FBt1/assignment_loss‚0.5304068306161623 1loss/SRqqt4/detection_loss‚2.6797009447835527 /loss/FRt4/detection_loss‚2.4491827960421397 1loss/SRqqt1/detection_loss‚2.0068417542641708 /loss/FRt3/assignment_loss‚3.679121147183811 2loss/SRqqt2/assignment_loss‚1.4107899493504343 2loss/SRqqt3/assignment_loss‚2.0049086631305757 /loss/FBt2/assignment_loss‚0.260787174098105 _step‚101 0loss/FRt2/assignment_loss‚1.9917734694062619 _runtime‚1146.650873056 /loss/FBt4/detection_loss‚2.8493446509043374 epoch‚2 % _timestamp‚1.7883903908950865e+09 /loss/FRt1/detection_loss‚2.3972484743908833 2loss/SRqqt1/assignment_loss‚1.8826956051853623 /loss/FBt1/detection_loss‚2.8463953009673526 2loss/SRqqt4/assignment_loss‚1.8932917354241856 /loss/FRt3/detection_loss‚2.8294900760931125e³S 2É ‚(:’ ÿ×âÔ§€¾u gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.967' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.593400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0r‰ß6Ê ‚(:« ÿ×âÔò—Ÿx disk.dm-1.in‚0.109375% disk./.usageGB‚1042.7632026672363% disk.nvme0n1.out‚5782.03759765625( proc.memory.availableMB‚ 2285.9453125 disk.sda2.in‚0' disk.nvme0c0n1.out‚5782.03759765625 disk.sda.in‚0.109375 disk.sda1.out‚0 cpu‚24.460011163339388 proc.cpu.threads‚17 network.sent‚ 2.237312e+06 disk.sda3.in‚0.109375 disk.rbd1.out‚ 0.67578125 disk.rbd2.out‚67.20947265625 disk.sda.out‚48.21875$ memory_percent‚86.04769706726074" proc.memory.rssMB‚ 2910.2578125 disk.rbd3.out‚ 1.36328125" disk.dm-0.out‚5782.03759765625 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.nvme0c0n1.in‚ 52.078125 disk.dm-1.out‚48.21875 disk.sda2.out‚0 disk.rbd3.in‚0 disk.sda3.out‚48.21875 disk.rbd1.in‚0 disk.nvme0n1.in‚ 52.078125 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0) disk./.usagePercent‚69.98071346760199 disk.dm-0.in‚ 52.078125 disk.sda1.in‚0 disk.drbd1777.out‚0) proc.memory.percent‚17.76280403137207 network.recv‚444217¨nÃË ‚(:’ ŽØâÔÂìªu gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.424' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.084799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0õœ¸2Ì ‚(:§ ŽØâÔä¾Íx disk.dm-1.in‚0.109375) proc.memory.percent‚17.76280403137207 disk.sda3.out‚ 48.859375) disk./.usagePercent‚69.98071551566225 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0$ memory_percent‚86.04865074157715 proc.cpu.threads‚17 network.sent‚ 2.32376e+06# disk.nvme0n1.out‚5787.857421875 disk.sda.out‚ 48.859375 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 0.67578125" proc.memory.rssMB‚ 2910.2578125 disk.rbd0.out‚0.078125 disk.rbd3.in‚0 disk.dm-1.out‚ 48.859375( proc.memory.availableMB‚ 2285.7890625 disk.rbd2.in‚0 disk.nvme0c0n1.in‚ 52.078125 network.recv‚449235 disk.rbd0.in‚0 disk.dm-0.in‚ 52.078125 disk.sda3.in‚0.109375 cpu‚24.474396201658585 disk.sda1.in‚0 disk.drbd1777.in‚0% disk.nvme0c0n1.out‚5787.857421875% disk./.usageGB‚1042.7632331848145 disk.rbd2.out‚69.41259765625 disk.nvme0n1.in‚ 52.078125 disk.sda.in‚0.109375 disk.dm-0.out‚5787.857421875 ßÖŠ’Í ‚(:‡ ØâÔŸíÐu gpu.0.gpu‚60.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.351' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.4702 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]\3?Î ‚(:´ ØâÔìúªx' disk.nvme0c0n1.out‚5886.93798828125 network.recv‚452993 disk.rbd0.in‚0 disk.drbd1777.out‚0 network.sent‚ 2.329223e+06 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.nvme0n1.in‚ 53.04296875! disk.nvme0c0n1.in‚ 53.04296875( disk./.usagePercent‚69.9807764454554 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚ 49.7421875 disk.drbd1777.in‚0 disk.rbd1.out‚ 0.67578125 disk.sda.in‚0.109375 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 53.04296875" disk.dm-0.out‚5886.93798828125% disk.nvme0n1.out‚5886.93798828125 disk.rbd0.out‚0.078125$ memory_percent‚86.04564666748047 disk.rbd1.in‚0 disk.rbd2.out‚70.42431640625 disk.sda1.in‚0 cpu‚24.488277320893808 disk.dm-1.out‚ 49.7421875 disk.rbd2.in‚0 disk.sda1.out‚0& proc.memory.availableMB‚ 2286.28125" proc.memory.rssMB‚ 2910.2578125) proc.memory.percent‚17.76280403137207% disk./.usageGB‚1042.7641410827637 disk.sda.out‚ 49.7421875 disk.sda2.out‚0 proc.cpu.threads‚17‹Ò!2Ï - _wandbruntime‚1187  _runtime‚1187q(O5ÆÐ À & lr-AdamW/pg2‚0.0016502040793575786 trainer/global_step‚1299 $ _timestamp‚1.788390431946685e+09 _runtime‚1187.702642401 _step‚102 & lr-AdamW/pg1‚0.0016502040793575786‰ŽœÊÑ Ä _runtime‚1187.702642401 _step‚102 & lr-AdamW/pg1‚0.0016502040793575786 & lr-AdamW/pg2‚0.0016502040793575786 trainer/global_step‚1299 $ _timestamp‚1.788390431946685e+09ftúñÇ2Ò - _wandbruntime‚1187  _runtime‚1187”Vý~Ò'Ó ÌO XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1064996412281 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27728183 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1118954535936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13647787 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚87439465453568 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88141964159488 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1070972736000 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1070776557568 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20606822 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 trainer/global_step‚1299 % _timestamp‚1.7883904319476845e+09 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚88510455228928 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚89260682860544 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚85319790249984 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1070972736000 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚89260943599104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1064804415745 _step‚103 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚88510455228928 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86384803701625 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚50003153 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚87439465453568 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1118718701056 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27705586 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚88510242011136 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27705586 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20607348 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6959038 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22274970 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚88510242011136 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834880 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚85319807289344 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚87439482492928 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1070776557568¡šž¼”&Ô ÐO ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1070972736000 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚89260943599104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1064804415745 _step‚103 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚88510455228928 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86384803701625 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚50003153 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚87439465453568 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1118718701056 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27705586 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚88510242011136 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27705586 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20607348 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6959038 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22274970 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚88510242011136 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834880 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚85319807289344 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚87439482492928 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1070776557568 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1064996412281 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27728183 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1118954535936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13647787 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚87439465453568 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88141964159488 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1070972736000 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1070776557568 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20606822 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 trainer/global_ste±"Ù2Bp‚1299 % _timestamp‚1.7883904319476845e+09 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚88510455228928 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚89260682860544 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚85319790249984g?V»12Õ - _wandbruntime‚1188  _runtime‚1188¸1jÛ&Ö ÕM I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚522 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1071771140096 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22291488 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27749081 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚50017946 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6964532 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚88207514559488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚50040571 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50040571 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13657818 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚525 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20621828 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚1299 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚88576079438848 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚87504504490496 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚89327321532928 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725248 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚88207489655808 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚85383406673920 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1119571151872 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1071574948352 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚87504521529856 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚89327060807680 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 $ _timestamp‚1.788390432696132e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚87504504490496 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1071574948352 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22291490 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚50017946 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22291490 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27726458 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚104 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20622353 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27749081 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1119806973440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13657821 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚86448987933253 _runtime‚1188.452855409†fÝß&× ÙM QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚88207489655808 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚85383406673920 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1119571151872 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1071574948352 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚87504521529856 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚89327060807680 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 $ _timestamp‚1.788390432696132e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚87504504490496 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1071574948352 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22291490 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚50017946 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22291490 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27726458 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚104 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20622353 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27749081 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1119806973440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13657821 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚86448987933253 _runtime‚1188.452855409 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚522 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1071771140096 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22291488 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27749081 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚50017946 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6964532 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚88207514559488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚50040571 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50040571 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13657818 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚525 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20621828 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚1299 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚88576079438848 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚87504504490496 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚89327321532928 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725248 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507h½¢÷O2Ø - _wandbruntime‚1188  _runtime‚1188 ØruXÙ Ò _runtime‚ 1188.45394396 2loss/SRqqt3/assignment_loss‚1.9239568983215884 1loss/FBt2/assignment_loss‚0.25411020228474224 1loss/SRqqt1/detection_loss‚1.7978910329418445 $loss/total_loss‚49.2070947389938 /loss/FRt1/assignment_loss‚3.627703741332081 2loss/SRqqt1/assignment_loss‚1.8900888914908838 /loss/FRt3/detection_loss‚2.4113231180019157 1loss/SRqqt2/detection_loss‚1.2745453887011693 /loss/FBt3/detection_loss‚2.4768367926279704 _step‚105 0loss/FRt2/assignment_loss‚2.3053905086697273 0loss/FRt3/assignment_loss‚3.5429169015828954 2loss/SRqqt4/assignment_loss‚2.6034772373162784 /loss/FBt4/assignment_loss‚0.545558529029436 .loss/FBt2/detection_loss‚0.647434796502485 0loss/FBt3/assignment_loss‚0.7020115380485853 1loss/SRqqt3/detection_loss‚2.1286519708403624 .loss/FBt1/detection_loss‚2.493738552555442 .loss/FBt4/detection_loss‚2.702611760181539 epoch‚2 % _timestamp‚1.7883904326981282e+09 1loss/FBt1/assignment_loss‚0.45157814896083437 .loss/FRt2/detection_loss‚1.886620478599139 /loss/FRt1/detection_loss‚2.2252889941228884 trainer/global_step‚1299 /loss/FRt4/assignment_loss‚4.768540847869146 2loss/SRqqt2/assignment_loss‚1.4857972429336412 /loss/FRt4/detection_loss‚2.5269989655131386 1loss/SRqqt4/detection_loss‚2.5340222005660715zgU\Ú Ö _runtime‚ 1188.45394396 2loss/SRqqt3/assignment_loss‚1.9239568983215884 1loss/FBt2/assignment_loss‚0.25411020228474224 1loss/SRqqt1/detection_loss‚1.7978910329418445 $loss/total_loss‚49.2070947389938 /loss/FRt1/assignment_loss‚3.627703741332081 2loss/SRqqt1/assignment_loss‚1.8900888914908838 /loss/FRt3/detection_loss‚2.4113231180019157 1loss/SRqqt2/detection_loss‚1.2745453887011693 /loss/FBt3/detection_loss‚2.4768367926279704 _step‚105 0loss/FRt2/assignment_loss‚2.3053905086697273 0loss/FRt3/assignment_loss‚3.5429169015828954 2loss/SRqqt4/assignment_loss‚2.6034772373162784 /loss/FBt4/assignment_loss‚0.545558529029436 .loss/FBt2/detection_loss‚0.647434796502485 0loss/FBt3/assignment_loss‚0.7020115380485853 1loss/SRqqt3/detection_loss‚2.1286519708403624 .loss/FBt1/detection_loss‚2.493738552555442 .loss/FBt4/detection_loss‚2.702611760181539 epoch‚2 % _timestamp‚1.7883904326981282e+09 1loss/FBt1/assignment_loss‚0.45157814896083437 .loss/FRt2/detection_loss‚1.886620478599139 /loss/FRt1/detection_loss‚2.2252889941228884 trainer/global_step‚1299 /loss/FRt4/assignment_loss‚4.768540847869146 2loss/SRqqt2/assignment_loss‚1.4857972429336412 /loss/FRt4/detection_loss‚2.5269989655131386 1loss/SRqqt4/detection_loss‚2.5340222005660715iþÕí‘Û ‚(:† ¬ØâÔ£Áµv gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚136.655' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.331 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0aò¸&=Ü ‚(:² ¬ØâÔõäy disk.sda1.out‚0 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 53.04296875 disk.rbd3.in‚0" disk.dm-0.out‚5888.49755859375 disk.rbd0.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda3.out‚ 50.171875$ memory_percent‚86.04998588562012) proc.memory.percent‚17.76280403137207 disk.rbd2.out‚72.63525390625 disk.rbd0.out‚0.078125 cpu‚24.50179784660857 proc.cpu.threads‚17( disk./.usagePercent‚69.9807726053424 disk.nvme0n1.in‚ 53.04296875 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 50.171875 disk.rbd1.in‚0 disk.rbd1.out‚ 0.67578125 disk.sda2.in‚0 disk.dm-0.in‚ 53.04296875 disk.sda3.in‚0.109375 network.sent‚ 2.335331e+06 network.recv‚457254 disk.sda.out‚ 50.171875( proc.memory.availableMB‚ 2285.5703125% disk.nvme0n1.out‚5888.49755859375 disk.sda.in‚0.109375' disk.nvme0c0n1.out‚5888.49755859375 disk.sda2.out‚0" proc.memory.rssMB‚ 2910.2578125% disk./.usageGB‚1042.7640838623047 disk.rbd2.in‚0HþK‘Ý ‚(:† »ØâÔë´v gpu.0.gpu‚22.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.0248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0|áç>Þ ‚(:³ »ØâÔ‡ãÔw proc.cpu.threads‚17 disk.sda.in‚0.109375 disk.sda2.out‚0 disk.dm-0.in‚ 54.1484375 disk.dm-1.in‚0.109375 disk.sda1.in‚0 disk.dm-1.out‚ 50.62890625 network.sent‚ 2.423102e+06 network.recv‚463893 disk.nvme0n1.in‚ 54.1484375 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.sda.out‚ 50.62890625 disk.sda2.in‚0 disk.rbd3.in‚0" disk.dm-0.out‚6467.78271484375) disk./.usagePercent‚70.06981484183405% disk.nvme0n1.out‚6467.78271484375 disk.rbd1.out‚ 0.67578125 disk.rbd2.in‚0 disk.rbd2.out‚73.64697265625" proc.memory.rssMB‚ 2910.2578125) proc.memory.percent‚17.76280403137207 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 54.1484375' disk.nvme0c0n1.out‚6467.78271484375% proc.memory.availableMB‚ 2285.8125$ disk./.usageGB‚1044.090877532959 disk.rbd1.in‚0$ memory_percent‚86.04850769042969 disk.sda3.out‚ 50.62890625 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.in‚0.109375 cpu‚24.515096801035373f¦Dcœß ‚(:‘ ÊØâԙܩu gpu.0.gpu‚57.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚94.112' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.822400000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‚[=Ã=à ‚(:² ÊØâÔâx disk.rbd0.in‚0 disk.sda3.in‚0.109375 disk.rbd1.in‚0 proc.cpu.threads‚17$ disk.nvme0n1.out‚7446.6220703125 disk.sda2.out‚0 disk.sda3.out‚ 51.37109375 disk.dm-1.in‚0.109375 cpu‚24.527645367909475 disk.drbd1777.in‚0 disk.dm-0.in‚ 56.953125! disk.dm-0.out‚7446.6220703125$ memory_percent‚86.04848384857178 disk.sda.out‚ 51.37109375& disk.nvme0c0n1.out‚7446.6220703125 disk.rbd3.in‚0) disk./.usagePercent‚70.06984914684364 disk.nvme0c0n1.in‚ 56.953125 disk.dm-1.out‚ 51.37109375 disk.rbd1.out‚ 0.67578125" proc.memory.rssMB‚ 2910.2578125) proc.memory.percent‚17.76280403137207 disk.sda.in‚0.109375 disk.rbd2.out‚76.16650390625% disk./.usageGB‚1044.0913887023926 disk.drbd1777.out‚0 network.recv‚467752 disk.nvme0n1.in‚ 56.953125 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.rbd2.in‚0) proc.memory.availableMB‚ 2285.81640625 network.sent‚ 2.428483e+06 disk.sda1.in‚0±ß2á - _wandbruntime‚1230  _runtime‚1230âéc¯Æâ À trainer/global_step‚1349 $ _timestamp‚1.788390474306246e+09 _runtime‚1230.062217982 _step‚106 & lr-AdamW/pg1‚0.0016401679731503501 & lr-AdamW/pg2‚0.0016401679731503501Ÿ\b?Êã Ä & lr-AdamW/pg1‚0.0016401679731503501 & lr-AdamW/pg2‚0.0016401679731503501 trainer/global_step‚1349 $ _timestamp‚1.788390474306246e+09 _runtime‚1230.062217982 _step‚106j-tÙ2ä - _wandbruntime‚1230  _runtime‚1230zý}å ÊO SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240029184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚91802354442752 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1104525735153 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14149484 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92579096399360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚91802141224960 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚88499776518144 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91418263879168 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚90691417299968 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _runtime‚ 1230.06345848 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚28775135 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23100968 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1110920103424 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14149487 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMº`żÌonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28752538 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21367500 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28775135 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932864 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91418238975488 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚88499759478784 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90691434339328 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚91802141224960 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1161097453056 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7218013 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 trainer/global_step‚1349 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚531 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21366969 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92579361332224 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1104717731689 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51876105 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23100968 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚51853506 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23100970 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚89604494249833 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788390474307249e+09 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚89604285213937 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚90691434339328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90691417299968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1110723924992 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚528 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1110920103424 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28752538 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _step‚107Ì.—Ô'æ ÎO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1110920103424 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28752538 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _step‚107 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240029184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚91802354442752 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1104525735153 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14149484 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92579096399360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚91802141224960 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚88499776518144 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91418263879168 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚90691417299968 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 _runtime‚ 1230.06345848 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚28775135 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23100968 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1110920103424 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14149487 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28752538 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21367500 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28775135 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932864 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91418238975488 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚88499759478784 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90691434339328 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚91802141224960 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1161097453056 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7218013 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 trainer/global_step‚1349 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚531 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21366969 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92579361332224 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1104717731689 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51876105 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23100968 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚51853506 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23100970 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚89604494249833 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788390474307249e+09 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚89604285213937 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚90691434339328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90691417299968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1110723924992 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚528 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243kYsí“2ç - _wandbruntime‚1230  _runtime‚1230Ó¸s3Û&è ÕM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1111718507520 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚88563358863360 trainer/global_step‚1349 _step‚108 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚28796033 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚92645464571392 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7223079 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92645725296640 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1105319618101 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1161935921152 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚90756473376256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51913523 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚90756456336896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725248 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7222543 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1161700099584 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883904750761335e+09 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1111522315776 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14159521 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚91868191883776 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1111718507520 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1105511614845 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28796033 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚90756456336896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21382061 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1111522315776 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14159518 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚536 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28773410 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚539 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51890898 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7223079 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92645725296640 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1105319618101 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1161935921152 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚90756473376256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51913523 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚90756456336896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260725248 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7222543 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1161700099584 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883904750761335e+09 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1111522315776 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14159521 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚91868191883776 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1111718507520 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1105511614845 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28796033 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚90756456336896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21382061 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1111522315776 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14159518 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚536 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28773410 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚539 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51890898 EDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚95094040438784 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7476185 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚91679728707584 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23926968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚532 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚95897658127360 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚92823975762145 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚93943369146368 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚92824184798041 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚529 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23926970 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29799490 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53749057 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚92824184798041 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚529 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23926970 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29799490 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53749057 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚95094040438784 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7476185 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚91679728707584 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23926968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚532 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚95897658127360 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚92823975762145 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936oërgK2÷ - _wandbruntime‚1273  _runtime‚1273ØÐàÒŒ ø ÕM PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚91743345131520 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29842985 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29820362 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚94008425222656 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94760039287808 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1204231268864 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚95160091097600 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53786475 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1145232934253 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1145040937509 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚95159877866496 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1151665874944 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚91743328092160 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 =7DeùWãÉOviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29820362 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 1273.07665736 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1151665874944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1203995447296 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22142232 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1151469683200 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚533 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14661221 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚53786475 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23943488 trainer/global_step‚1399 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚94008408183296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _step‚112 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7481014 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94760064191488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896Nµ—4ß&ù ÙM SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 1273.07665736 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1151665874944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1203995447296 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22142232 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1151469683200 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚533 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14661221 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚53786475 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23943488 trainer/global_step‚1399 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚94008408183296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _step‚112 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7481014 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94760064191488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚91743345131520 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29842985 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29820362 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚94008425222656 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94760039287808 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1204231268864 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚95160091097600 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53786475 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1145232934253 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1145040937509 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚95159877866496 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1151665874944 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚91743328092160 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29820362 ODeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚96043666310353 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1190814838272 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835712 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97970813511168 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1245351495680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1184160370505 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚55599410 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30846442 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15152887 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1245589427712 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1190618659840 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1183968373969 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1190814838272 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 % _timestamp‚1.7883905587631803e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚99216140103168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1449 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚544 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚97195320992768 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237932032 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24752968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚99216402938880 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚98385939652608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚30869039 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚98385939652608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚98386152870400 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _runtime‚1314.519400666 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚98386152870400 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚30869039 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚97195338032128 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15152884 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24752970 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7735636 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7735095 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24752970 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30846442 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive¨œê6Ñ_split_bytes.large_pool.freed‚97970788607488 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚96043875346249 _step‚115 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1190618659840 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22887979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚55622009 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24752968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚99216402938880 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚98385939652608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚30869039 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚98385939652608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚98386152870400 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _runtime‚1314.519400666 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚98386152870400 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚30869039 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚97195338032128 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15152884 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24752970 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7735636 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7735095 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24752970 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30846442 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚97970788607488 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚96043875346249 _step‚115 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1190618659840 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22887979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚55622009 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚96043666310353 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1190814838272 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835712 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97970813511168 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1245351495680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1184160370505 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚55599410 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30846442 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15152887 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1245589427712 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1190618659840 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1183968373969 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1190814838272 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 % _timestamp‚1.7883905587631803e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚99216140103168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1449 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚544 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚97195320992768 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0s¯ú·i2‰ - _wandbruntime‚1315  _runtime‚1315¶`*SÜ&Š ÖM BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30867314 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚98036339007488 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1246201891840 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚97260377069056 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15162921 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1246441907712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 % _timestamp‚1.7883905595253835e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24769490 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚55659427 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚98036314103808 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚1449 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _step‚116 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚55636802 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1191613242368 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22903092 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1191613242368 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚98451990311424 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15162918 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚98451990311424 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1184954253661 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚99282780915200 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚96108268613981 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚98451777080320 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚55636802 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22903636 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚96108059577877 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚94923314360320 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚55659427 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7740174 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _runtime‚1315.281860806 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0ø¾¸;à&‹ ÚM S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚98451990311424 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1184954253661 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚99282780915200 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚96108268613981 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚98451777080320 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚55636802 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22903636 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚96108059577877 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚94923314360320 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚55659427 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7740174 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _runtime‚1315.281860806 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30867314 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚98036339007488 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1246201891840 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚97260377069056 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15162921 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1246441907712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 % _timestamp‚1.7883905595253835e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24769490 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚55659427 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚98036314103808 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚1449 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _step‚116 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚55636802 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1191613242368 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22903092 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1191613242368 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚98451990311424 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15162918t3c¶2Œ - _wandbruntime‚1315  _runtime‚1315„ Ï trainer/global_step‚1449 % _timestamp‚1.7883905595271091e+09 1loss/SRqqt2/assignment_loss‚1.416407602813302 1loss/SRqqt4/assignment_loss‚2.286759049321214 /loss/FBt1/detection_loss‚3.0596241818534002 /loss/FBt2/detection_loss‚0.60182Æg±jM43319820613 _runtime‚1315.282927707 _step‚117 /loss/FRt3/assignment_loss‚3.936171569019915 .loss/FRt3/detection_loss‚2.562951224755092 1loss/SRqqt4/detection_loss‚2.5953245812820063 epoch‚2 2loss/SRqqt1/assignment_loss‚1.8759588122777981 1loss/FBt2/assignment_loss‚0.13951445409724123 0loss/SRqqt2/detection_loss‚0.965063930903044 1loss/FBt4/assignment_loss‚0.16236646127487933 1loss/FBt3/assignment_loss‚0.48646151001254717 /loss/FRt1/detection_loss‚2.5794410508039385 1loss/SRqqt1/detection_loss‚1.8161230929400942 1loss/SRqqt3/detection_loss‚2.1706873930162853 .loss/FBt4/detection_loss‚3.589638897350856 /loss/FRt2/assignment_loss‚2.386780329269938 /loss/FBt1/assignment_loss‚0.407078467188209 $loss/total_loss‚49.7238483436126 0loss/FRt1/assignment_loss‚3.9948646440062414 /loss/FRt4/assignment_loss‚4.417377719512353 2loss/SRqqt3/assignment_loss‚1.6830545504060057 .loss/FRt2/detection_loss‚1.779903674332222 .loss/FRt4/detection_loss‚2.497267810197977 .loss/FBt3/detection_loss‚2.313203004995982!Ë2ØYŽ Ó 2loss/SRqqt1/assignment_loss‚1.8759588122777981 1loss/FBt2/assignment_loss‚0.13951445409724123 0loss/SRqqt2/detection_loss‚0.965063930903044 1loss/FBt4/assignment_loss‚0.16236646127487933 1loss/FBt3/assignment_loss‚0.48646151001254717 /loss/FRt1/detection_loss‚2.5794410508039385 1loss/SRqqt1/detection_loss‚1.8161230929400942 1loss/SRqqt3/detection_loss‚2.1706873930162853 .loss/FBt4/detection_loss‚3.589638897350856 /loss/FRt2/assignment_loss‚2.386780329269938 /loss/FBt1/assignment_loss‚0.407078467188209 $loss/total_loss‚49.7238483436126 0loss/FRt1/assignment_loss‚3.9948646440062414 /loss/FRt4/assignment_loss‚4.417377719512353 2loss/SRqqt3/assignment_loss‚1.6830545504060057 .loss/FRt2/detection_loss‚1.779903674332222 .loss/FRt4/detection_loss‚2.497267810197977 .loss/FBt3/detection_loss‚2.313203004995982 trainer/global_step‚1449 % _timestamp‚1.7883905595271091e+09 1loss/SRqqt2/assignment_loss‚1.416407602813302 1loss/SRqqt4/assignment_loss‚2.286759049321214 /loss/FBt1/detection_loss‚3.0596241818534002 /loss/FBt2/detection_loss‚0.6018243319820613 _runtime‚1315.282927707 _step‚117 /loss/FRt3/assignment_loss‚3.936171569019915 .loss/FRt3/detection_loss‚2.562951224755092 1loss/SRqqt4/detection_loss‚2.5953245812820063 epoch‚2u¦oÎ[‘ ‚(:† ¤ÙâÔô¤Äu gpu.0.gpu‚35.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚92.648' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.5296 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0hÔé>B ‚(:· ¤ÙâÔµÜÝx disk.sda2.out‚0 network.sent‚ 2.625097e+06 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.rbd2.out‚88.79150390625 disk.rbd0.in‚0 disk.sda2.in‚0$ memory_percent‚86.05141639709473) disk./.usagePercent‚70.06959953949774 disk.nvme0n1.in‚ 58.36328125 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0& disk.nvme0c0n1.out‚7852.9599609375 network.recv‚495491 disk.dm-1.out‚ 54.95703125 disk.sda.in‚0.109375 disk.drbd1777.out‚0( proc.memory.availableMB‚ 2285.3359375) proc.memory.percent‚17.76280403137207 cpu‚24.598895018625104 proc.cpu.threads‚17 disk.sda3.out‚ 54.95703125 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 58.36328125 disk.dm-0.in‚ 58.36328125% disk./.usageGB‚1044.0876693725586 disk.rbd3.in‚0! disk.dm-0.out‚7852.9599609375" proc.memory.rssMB‚ 2910.2578125 disk.rbd1.out‚ 0.67578125 disk.rbd2.in‚0$ disk.nvme0n1.out‚7852.9599609375 disk.sda.out‚ 54.95703125 disk.rbd3.out‚ 1.36328125dÛ;‘ ‚(:’ ³ÙâÔ¥Ôu gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.256' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.051199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.Åì $’ ‚(:™ ³ÙâÔþ»Ïx disk.drbd1777.in‚0 disk.sda2.in‚0' disk.nvme0c0n1.out‚7855.71044921875 network.recv‚501503) disk./.usagePercent‚70.06953758567445 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd2.in‚0% disk.nvme0n1.out‚7855.71044921875 disk.dm-0.in‚62.3125 disk.rbd0.in‚0# memory_percent‚86.0504150390625 disk.dm-1.out‚55.8125 disk.rbd2.out‚91.87353515625 disk.nvme0c0n1.in‚62.3125 disk.rbd3.out‚ 1.36328125" disk.dm-0.out‚7855.71044921875 disk.nvme0n1.in‚62.3125 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.rbd3.in‚0 disk.sda1.in‚0 network.sent‚ 2.71272e+06 disk.rbd1.out‚ 0.67578125 disk.drbd1777.out‚0" proc.memory.rssMB‚ 2910.2578125) proc.memory.percent‚17.76280403137207 cpu‚24.60985275008077 disk.rbd1.in‚0% disk./.usageGB‚1044.0867462158203 disk.sda3.out‚55.8125 disk.sda.in‚0.109375 disk.sda.out‚55.8125 disk.sda2.out‚0" proc.memory.availableMB‚2285.5 proc.cpu.threads‚17ªøâj’“ ‚(:‡ ÂÙâÔ—ÚÝu gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚110.854' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.1708 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ka6” ‚(:« ÂÙâÔó¦ôx network.sent‚ 2.71804e+06 disk.sda.out‚ 56.2734375 disk.sda1.out‚0 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.rbd2.out‚93.97509765625 cpu‚24.620741693985572 disk.rbd0.in‚0 disk.sda3.out‚ 56.2734375 disk.rbd1.out‚ 0.67578125 disk.rbd2.in‚0" proc.memory.rssMB‚ 2910.2578125 disk.nvme0n1.in‚ 62.3203125 disk.drbd1777.in‚0 disk.dm-0.out‚7985.677734375' proc.memory.availableMB‚ 2285.734375 proc.cpu.threads‚17) disk./.usagePercent‚70.07135882327304 disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚ 62.3203125$ memory_percent‚86.04898452758789 network.recv‚505344# disk.nvme0n1.out‚7985.677734375 disk.dm-1.out‚ 56.2734375 disk.rbd1.in‚0 disk.drbd1777.out‚0$ disk./.usageGB‚1044.113883972168 disk.sda.in‚0.109375 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 62.3203125 disk.sda3.in‚0.109375) proc.memory.percent‚17.76280403137207% disk.nvme0c0n1.out‚7985.677734375 disk.rbd3.in‚0Œ )@2• - _wandbruntime‚1356  _runtime‚1356×T – ¼ $ lr-AdamW/pg1‚0.00160618403979925 $ lr-AdamW/pg2‚0.00160618403979925 trainer/global_step‚1499 $ _timestamp‚1.788390600793101e+09 _runtime‚1356.549425682 _step‚118o²kþÆ— À $ lr-AdamW/pg1‚0.00160618403979925 $ lr-AdamW/pg2‚0.00160618403979925 trainer/global_step‚1499 $ _timestamp‚1.788390600793101e+09 _runtime‚1356.549425682 _step‚118vÔe>2˜ - _wandbruntime‚1356  _runtime‚1356·O¢Þ'™ ØO LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31893394 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚101677838866432 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚102534618652160 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25578968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25578968 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚57472362 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚99263565894457 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25578970 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚101677838866432 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244223488 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23648588 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23649098 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚100447289878528 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚102534349524992 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7994511 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269127168 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25578970 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31915991 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚100447289878528 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚98039667165184 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1230566027264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚510 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚101247088327168 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1223881689913 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1230566027264 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15654584 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚507 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883906007939036e+09 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130u1‹åâ'š ÜO JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _step‚119 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚1356.550127724 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31893394 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚100447272839168 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23648588 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23649098 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚100447289878528 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚102534349524992 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7994511 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269127168 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25578970 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31915991 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚100447289878528 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚98039667165184 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1230566027264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚510 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚101247088327168 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1223881689913 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1230566027264 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15654584 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚507 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883906007939036e+09 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31893394 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚101677838866432 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚102534618652160 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25578968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25578968 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚57472362 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚99263565894457 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25578970 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚101677838866432 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244223488 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚98103266549760 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1231560609792 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57509754 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚101743676294144 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚100512328915456 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚100512311876096 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15664618 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1231364418048 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚25595490 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25595488 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1288122569728 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1224675573069 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57532379 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1231560609792 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1288364682752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1224483576325 _runtime‚1357.305094019 trainer/global_step‚1499 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23663689 \HDeviceStatsMo>Îh8nitor.on_train_batch_end/inactive_split_bytes.all.allocated‚102600978506240 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57509754 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚101743889525248 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57532379 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚102600711489536 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31914266 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25595488 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016704 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1231364418048 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚101312588919808 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883906015487332e+09 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚101743889525248 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31936889 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31914266 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23664213 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25595490 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3±dR1ì& æM G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57509754 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚101743676294144 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚100512328915456 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚100512311876096 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15664618 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1231364418048 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚25595490 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25595488 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1288122569728 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1224675573069 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57532379 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1231560609792 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1288364682752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1224483576325 _runtime‚1357.305094019 trainer/global_step‚1499 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23663689 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚102600978506240 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57509754 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚101743889525248 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57532379 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚102600711489536 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31914266 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25595488 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016704 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1231364418048 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚101312588919808 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883906015487332e+09 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚101743889525248 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31936889 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31914266 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23664213 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25595490 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚98103266549760 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1231560609792 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243x617¹2ž - _wandbruntime‚1357  _runtime‚1357¥î ÷\Ÿ Ö 2loss/SRqqt1/assignment_loss‚1.7387385973790745 /loss/FBt3/assignment_loss‚0.885386945393223 1loss/FBt4/assignment_loss‚0.31855360895056756 /loss/FBt1/detection_loss‚3.0213245261799204 % _timestamp‚1.7883906015504093e+09 1loss/SRqqt2/detection_loss‚0.9775191037180392 /loss/FBt2/detection_loss‚0.6636239417595788 epoch‚2 0loss/FRt2/assignment_loss‚2.1236086711971036 /loss/FRt2/detection_loss‚1.9127595495056668 .loss/FBt4/detection_loss‚2.389630502776096 2loss/SRqqt3/assignment_loss‚1.8139936166662205 /loss/FRt4/detection_loss‚2.3719947792234874 1loss/SRqqt3/detection_loss‚2.2611671889884564 1loss/SRqqt4/detection_loss‚2.4459193362032665 _runtime‚1357.306215613 0loss/FRt3/assignment_loss‚3.5099565498530865 /loss/FRt4/assignment_loss‚4.167005604221707 /loss/FRt3/detection_loss‚2.8326478889212012 0loss/FRt1/assignment_loss‚3.8108753801621114 2loss/SRqqt2/assignment_loss‚1.3762759767031592 0loss/FBt1/assignment_loss‚0.3556462811062705 1loss/FBt2/assignment_loss‚0.14577299235339897 /loss/FRt1/detection_loss‚2.4546281277518913 /loss/FBt3/detection_loss‚2.7517424730154185 _step‚121 1loss/SRqqt4/assignment_loss‚2.049584102104692 1loss/SRqqt1/detection_loss‚1.8591791847804646 $loss/total_loss‚48.2375349289141 trainer/global_step‚1499ë÷úÖ`  Ú 0loss/FRt1/assignment_loss‚3.8108753801621114 2loss/SRqqt2/assignment_loss‚1.3762759767031592 0loss/FBt1/assignment_loss‚0.3556462811062705 1loss/FBt2/assignment_loss‚0.14577299235339897 /loss/FRt1/detection_loss‚2.4546281277518913 /loss/FBt3/detection_loss‚2.7517424730154185 _step‚121 1loss/SRqqt4/assignment_loss‚2.049584102104692 1loss/SRqqt1/detection_loss‚1.8591791847804646 $loss/total_loss‚48.2375349289141 trainer/global_step‚1499 2loss/SRqqt1/assignment_loss‚1.7387385973790745 /loss/FBt3/assignment_loss‚0.885386945393223 1loss/FBt4/assignment_loss‚0.31855360895056756 /loss/FBt1/detection_loss‚3.0213245261799204 % _timestamp‚1.7883906015504093e+09 1loss/SRqqt2/detection_loss‚0.9775191037180392 /loss/FBt2/detection_loss‚0.6636239417595788 epoch‚2 0loss/FRt2/assignment_loss‚2.1236086711971036 /loss/FRt2/detection_loss‚1.9127595495056668 .loss/FBt4/detection_loss‚2.389630502776096 2loss/SRqqt3/assignment_loss‚1.8139936166662205 /loss/FRt4/detection_loss‚2.3719947792234874 1loss/SRqqt3/detection_loss‚2.2611671889884564 1loss/SRqqt4/detection_loss‚2.4459193362032665 _runtime‚1357.306215613 0loss/FRt3/assignment_loss‚3.5099565498530865 /loss/FRt4/assignment_loss‚4.167005604221707 /loss/FRt3/detection_loss‚2.8326478889212012yë"5’¡ ‚(:‡ ÑÙâÔ–³±u gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.087' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.4174 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0£ =?¢ ‚(:´ ÑÙâÔ«ÑÝw$ disk./.usageGB‚1044.095043182373 disk.dm-1.out‚ 57.171875% disk.nvme0n1.out‚8000.47802734375 disk.sda.in‚0.109375 disk.dm-1.in‚0.109375! disk.nvme0c0n1.in‚ 63.39453125 disk.sda2.in‚0" disk.dm-0.out‚8000.47802734375 cpu‚24.63111985669239 proc.cpu.threads‚17 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.sda3.out‚ 57.171875 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125) proc.memory.percent‚17.76282787322998 network.sent‚ 2.723367e+06 disk.rbd1.out‚ 0.67578125 disk.sda1.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 2910.26171875$ memory_percent‚86.05067729949951 network.recv‚508988 disk.rbd2.out‚97.06103515625 disk.nvme0n1.in‚ 63.39453125 disk.sda.out‚ 57.171875 disk.drbd1777.in‚0 disk.drbd1777.out‚0) proc.memory.availableMB‚ 2285.45703125 disk.dm-0.in‚ 63.39453125) disk./.usagePercent‚70.07009440206143 disk.rbd0.in‚0' disk.nvme0c0n1.out‚8000.47802734375Â8º…£ ‚(:’ àÙâÔ£±¡u gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.383' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.076600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0;5@¤ ‚(:µ àÙâÔ­úŠz disk.rbd1.out‚ 0.67578125 disk.sda2.out‚0 disk.sda3.in‚0.109375 disk.sda1.out‚0$ memory_percent‚86.05213165283203# proc.memory.rssMB‚ 2910.26171875) proc.memory.percent‚17.76282787322998 disk.rbd1.in‚0 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd3.in‚0 network.sent‚ 2.810667e+06 disk.nvme0n1.in‚ 63.39453125 disk.sda.out‚ 57.4765625& proc.memory.availableMB‚ 2285.21875) disk./.usagePercent‚70.07084527215936! disk.nvme0c0n1.in‚ 63.39453125' disk.nvme0c0n1.out‚8112.44091796875 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125% disk./.usageGB‚1044.1062316894531 cpu‚24.6416713398459 disk.dm-1.out‚ 57.4765625% disk.nvme0n1.out‚8112.44091796875" disk.dm-0.out‚8112.44091796875 disk.drbd1777.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚ 57.4765625 disk.dm-1.in‚0.109375 proc.cpu.threads‚17 network.recv‚515247 disk.dm-0.in‚ 63.39453125! disk.rbd2.out‚101.31103515625ôgôõ‘¥ ‚(:† ïÙâÔ´Á u gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚95.596' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.1192 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±)ü:¦ ‚(:¯ ïÙâÔöáõx network.recv‚519748 disk.sda2.out‚0 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.sda.out‚58.09375% disk./.usageGB‚1044.0930786132812 disk.rbd2.in‚0! disk.rbd2.out‚106.43994140625) proc.memory.percent‚17.76282787322998' disk.nvme0c0n1.out‚8213.92138671875" disk.dm-0.out‚8213.92138671875 disk.sda3.out‚58.09375 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.67578125% disk.nvme0n1.out‚8213.92138671875$ memory_percent‚86.92455291748047# proc.memory.rssMB‚ 2910.26171875 disk.drbd1777.in‚0 cpu‚24.65179892754493 proc.cpu.threads‚17 disk.sda1.out‚0( disk./.usagePercent‚70.0699625581813! disk.nvme0c0n1.in‚ 64.11328125 disk.rbd3.in‚0 disk.dm-0.in‚ 64.11328125 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.nvme0n1.in‚ 64.11328125 network.sent‚ 2.817204e+06 disk.rbd3.out‚ 1.36328125 disk.dm-1.out‚58.09375 disk.rbd0.out‚0.078125& proc.memory.availableMB‚ 2142.28125 disk.sda1.in‚0Œ>§’2§ - _wandbruntime‚1398  _runtime‚1398· {Ǩ Á & lr-AdamW/pg2‚0.0015935957182532736 trainer/global_step‚1549 % _timestamp‚1.7883906425702827e+09 _runtime‚1398.326253489 _step‚122 & lr-AdamW/pg1‚0.0015935957182532736ã+žªË© Å & lr-AdamW/pg1‚0.0015935957182532736 & lr-AdamW/pg2‚0.0015935957182532736 trainer/global_step‚1549 % _timestamp‚1.7883906425702827e+09 _runtime‚1398.326253489 _step‚122zÊëì2ª - _wandbruntime‚1398  _runtime‚1398Ç¥'&« ÜO \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚103699224685568 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26404970 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚105852923914240 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32940346 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16156284 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚104969951298048 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚509 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883906425713441e+09 _step‚123 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1270709573120 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1329560771072 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1263411012785 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚103699241724928 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚104969738080256 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚102483256442665 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚26404968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚32962943 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚102483047406769 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32940346 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8252108 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚104523338239488 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚1398.327540113 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚104523363143168 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26404970 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚103699241724928 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1329318644736 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚104969738080256 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8252617 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24408904 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚101219636393984 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24408392 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 trainer/global_step‚1549 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1270709573120 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1270513394688 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16156287 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor/{wyÓ.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚26404968 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432`<ÜÅæ'¬ àO ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26404970 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚103699241724928 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1329318644736 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚104969738080256 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8252617 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24408904 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚101219636393984 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24408392 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 trainer/global_step‚1549 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1270709573120 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1270513394688 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16156287 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚26404968 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚103699224685568 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26404970 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚105852923914240 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32940346 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16156284 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚104969951298048 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚509 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883906425713441e+09 _step‚123 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1270709573120 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1329560771072 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1263411012785 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚103699241724928 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚104969738080256 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚102483256442665 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚26404968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚32962943 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚102483047406769 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32940346 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8252108 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚104523338239488 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚1398.327540113 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚104523363143168 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0{%áä2­ - _wandbruntime‚1399  _runtime‚1399_Qì&® æM >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1330395149312 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚59382706 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1271311785472 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59405331 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26421488 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26421488 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚104588888639488 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚1549 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26421490 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1264204895733 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32983841 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚101283235778560 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24423479 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59382706 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32961218 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16166318 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚102547649710397 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚103764263722496 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1271507977216 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚103764280761856 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚103764263722496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26421490 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚105919283788800 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1271311785472 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59405331 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32961218 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚542 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8257161 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1271507977216 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24424021 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚105919004189184 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1330140453376ÅÖÉuð&¯ êM [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚103764280761856 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59405331 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32961218 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚542 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8257161 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1271507977216 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24424021 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚105919004189184 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1330140453376 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1330395149312 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚59382706 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1271311785472 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59405331 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26421488 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26421488 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚104588888639488 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚1549 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26421490 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1264204895733 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32983841 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚101283235778560 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24423479 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59382706 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32961218 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16166318 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚102547649710397 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚103764263722496 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1271507977216 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚103764280761856 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚103764263722496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26421490 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚105919283788800 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1271311785472 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 EDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883906849949775e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚108261637294080 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚106951193571328 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _runtime‚ 1440.75119994 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚106951193571328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1310460762112 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932864 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚104399605622784 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1310656940544 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚108261850511872 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚541 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚109171315717632 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25168138 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33987298 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1371677758464 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚105702946990873 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚33987298 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16657984 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚107799637959168 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25168682 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚1599 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚106951176531968 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1371437729280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61218266 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8510695 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚108261637294080 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750¡¥Ê­å'¾ ßO ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚106951193571328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1310460762112 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932864 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚104399605622784 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1310656940544 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚108261850511872 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚541 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚109171315717632 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25168138 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33987298 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1371677758464 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚105702946990873 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚33987298 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16657984 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚107799637959168 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25168682 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚1599 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚106951176531968 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1371437729280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61218266 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8510695 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚108261637294080 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27230968 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1303324328729 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚107799613055488 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚104399622662144 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883906849949775e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚108261637294080 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚106951193571328 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _runtime‚ 1440.75119994dvW2¿ - _wandbruntime‚1441  _runtime‚1441Ue SÀ æM UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1303926215141 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚16668021 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27247488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34008170 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚107016232608256 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61278283 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚522 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚108327687952896 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚107865163455488 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚104463205007360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61278283 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27247490 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚109237671576064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚107016232608256 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚108327687952896 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚107016215568896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1311455344640 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25183213 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1372289076224 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1311259152896 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8515195 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚105767340258605 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34030793 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚1441.521645333 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚105767131222501 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚109237427628032 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1304118211885 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883906857648187e+09 _step‚128 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27247488(OKzñ&Á ëM UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25183213 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1372289076224 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1311259152896 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8515195 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚105767340258605 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34030793 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚1441.521645333 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚105767131222501 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚109237427628032 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1304118211885 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883906857648187e+09 _step‚128 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27247488 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1303926215141 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚16668021 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27247488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34008170 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚107016232608256 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61278283 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚522 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚108327687952896 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚107865163455488 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚104463205007360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61278283 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27247490 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚109237671576064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚107016232608256 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚108327687952896 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚107016215568896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1311455344640 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35034250 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚524 _runtime‚1482.947368138 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8768727 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚112489732745216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚110203145417728 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1350604307968 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63113817 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28056970 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚107579591890944 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚111553749725696 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚111553536507904 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35034250 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1350408129536 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚107579574851584 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚110203145417728 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1350604307968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28056968 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1350408129536 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28056970 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63091218 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35056847 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17159687 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚111075887871488 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1342853651601 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17159684 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108922428503185 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25928414 trainer/global_step‚1649 $ _timestamp‚1.788390727191159e+09 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚112489461520896 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1413819970048 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1413573649408 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750Ì•»£± Î àO IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35034250 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚524 _runtime‚1482.947368138 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8768727 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚112489732745216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345549 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚110203145417728 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1350604307968 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63113817 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28056970 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚107579591890944 VBDeviceStatsMonitor.on_train_b&´95atch_start/active_bytes.all.allocated‚111553749725696 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚111553536507904 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35034250 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1350408129536 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚107579574851584 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚110203145417728 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1350604307968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28056968 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1350408129536 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28056970 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63091218 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35056847 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17159687 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚111075887871488 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1342853651601 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17159684 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108922428503185 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25928414 trainer/global_step‚1649 $ _timestamp‚1.788390727191159e+09 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚112489461520896 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1413819970048 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1413573649408 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚131 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚111553749725696 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271224320 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚111075912775168 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25927890 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0ƒôNÕ•2Ï - _wandbruntime‚1483  _runtime‚1483sê&Ð äM J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚111141413367808 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚110268184454656 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚111619587166720 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35055122 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28073490 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚108986821770709 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1351402712064 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28073488 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25942934 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233724416 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63151235 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 $ _timestamp‚1.788390727968801e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63151235 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8773739 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1351206520320 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1343647534549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28073488 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17169721 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚112556091732992 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63128610 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚110268167415296 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚110268184454656 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚111619587166720 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚111619373935616 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1343839531293 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1351206520320 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚107643191275520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚1649 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8773216 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108987030806813 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚111141438271488 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚132 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35055122 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1351402712064 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580¦é"ï&Ñ éM UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8773739 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1351206520320 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1343647534549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28073488 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17169721 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚112556091732992 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63128610 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚110268167415296 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚110268184454656 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚111619587166720 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚111619373935616 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1343839531293 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1351206520320 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚107643191275520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚1649 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8773216 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108987030806813 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚111141438271488 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚132 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35055122 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1351402712064 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚111141413367808 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚110268184454656 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚111619587166720 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35055122 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28073490 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚108986821770709 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1351402712064 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28073488 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25942934 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233724416 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63151235 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 $ _timestamp‚1.788390727968801e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63151235 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0„ f&s2Ò - _wandbruntime‚1483  _runtime‚1483„>ºšXÓ Ò % _timestamp‚1.7883907279710376e+09 2loss/SRqqt1/assignment_loss‚1.8558708911815893 1loss/SRqqt4/assignment_loss‚2.227404727148158 1loss/FBt4/assignment_loss‚0.22464439177009113 0loss/SRqqt4/detection_loss‚2.512111858810697 1loss/SRqqt2/detection_loss‚1.0636147566315697 /loss/FBt1/detection_loss‚2.6975988285108046 trainer/global_step‚1649 _runtime‚1483.726903654 0loss/FBt2/assignment_loss‚0.2626821080873237 .loss/FRt3/detection_loss‚2.505603677696652 .loss/FBt3/detection_loss‚2.659073850938252 epoch‚2 /loss/FRt1/detection_loss‚2.5438768360121498 2loss/SRqqt2/assignment_loss‚1.2460663829614798 2loss/SRqqt3/assignment_loss‚1.8920625220326817 /loss/FRt1/assignment_loss‚3.942128378769447 /loss/FRt3/assignment_loss‚3.358414889117818 /loss/FRt4/assignment_loss‚4.430246673311506 0loss/FBt3/assignment_loss‚0.5746494924782642 /loss/FRt2/detection_loss‚1.7550742342927546 1loss/SRqqt1/detection_loss‚2.0097146746024346 1loss/SRqqt3/detection_loss‚1.9188365115254533 %loss/total_loss‚47.80952857738029 /loss/FRt2/assignment_loss‚2.025804839044247 0loss/FBt1/assignment_loss‚0.3064142231067473 /loss/FBt4/detection_loss‚2.7017835168277515 _step‚133 /loss/FRt4/detection_loss‚2.5430599893842425 /loss/FBt2/detection_loss‚0.5527903231381739ðý‚E]Ô × /loss/FRt1/assignment_loss‚3.942128378769447 /loss/FRt3/assignment_loss‚3.358414889117818 /loss/FRt4/assignment_loss‚4.430246673311506 0loss/FBt3/assignment_loss‚0.5746494924782642 /loss/FRt2/detection_loss‚1.7550742342927546 1loss/SRqqt1/detection_loss‚2.0097146746024346 1loss/SRqqt3/detection_loss‚1.9188365115254533 %loss/total_loss‚47.80952857738029 /loss/FRt2/assignment_loss‚2.025804839044247 0loss/FBt1/assignment_loss‚0.3064142231067473 /loss/FBt4/detection_loss‚2.7017835168277515 _step‚133 /loss/FRt4/detection_loss‚2.5430599893842425 /loss/FBt2/detection_loss‚0.5527903231381739 % _timestamp‚1.7883907279710376e+09 2loss/SRqqt1/assignment_loss‚1.8558708911815893 1loss/SRqqt4/assignment_loss‚2.227404727148158 1loss/FBt4/assignment_loss‚0.22464439177009113 0loss/SRqqt4/detection_loss‚2.512111858810697 1loss/SRqqt2/detection_loss‚1.0636147566315697 /loss/FBt1/detection_loss‚2.6975988285108046 trainer/global_step‚1649 _runtime‚1483.726903654 0loss/FBt2/assignment_loss‚0.2626821080873237 .loss/FRt3/detection_loss‚2.505603677696652 .loss/FBt3/detection_loss‚2.659073850938252 epoch‚2 /loss/FRt1/detection_loss‚2.5438768360121498 2loss/SRqqt2/assignment_loss‚1.2460663829614798 2loss/SRqqt3/assignment_loss‚1.8920625220326817…ŒÚ»œÕ ‚(:… ÉÚâÔ³Ôªu gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚132.13' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.426 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0h’Öš@Ö ‚(:µ ÉÚâÔÏðøw disk.drbd1777.out‚0" disk.dm-0.out‚8621.08056640625 disk.sda3.out‚ 61.3203125! disk.rbd2.out‚116.82373046875 disk.nvme0n1.in‚ 64.4921875 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.dm-0.in‚ 64.4921875 disk.dm-1.out‚ 61.3203125 disk.sda.out‚ 61.3203125$ proc.memory.availableMB‚2141.625# proc.memory.rssMB‚ 2910.26171875 proc.cpu.threads‚17 disk.drbd1777.in‚0% disk.nvme0n1.out‚8621.08056640625 disk.nvme0c0n1.in‚ 64.4921875' disk.nvme0c0n1.out‚8621.08056640625 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.sda2.in‚0 network.sent‚ 3.013805e+06 network.recv‚546949% disk./.usageGB‚1044.0959510803223 disk.rbd1.out‚ 0.91943359375$ memory_percent‚86.92855834960938 disk.rbd0.in‚0) proc.memory.percent‚17.76282787322998 disk.rbd2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.rbd1.in‚0 cpu‚24.708039512344495) disk./.usagePercent‚70.07015533185458 disk.sda1.in‚0wù†Îœ× ‚(:‘ ØÚâÔúÕ¢u gpu.0.gpu‚35.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.372' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.874399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0k– Ø ‚(:½ ØÚâÔ“ñ™x disk.dm-1.in‚0.109375 disk.sda.out‚ 61.80859375) proc.memory.availableMB‚ 2141.81640625 cpu‚24.716682374072036% disk./.usageGB‚1044.0960273742676' disk.nvme0c0n1.out‚8623.47900390625 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.nvme0n1.in‚ 64.4921875 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚ 64.4921875 disk.sda2.in‚0% disk.nvme0n1.out‚8623.47900390625 disk.sda3.out‚ 61.80859375 disk.rbd1.out‚ 0.91943359375 disk.sda1.out‚0# proc.memory.rssMB‚ 2910.2617187586«Å( disk.sda1.in‚0" disk.dm-0.out‚8623.47900390625 network.sent‚ 3.101192e+06) disk./.usagePercent‚70.07016045200525 disk.drbd1777.out‚0 disk.rbd0.out‚0.078125$ memory_percent‚86.92739009857178) proc.memory.percent‚17.76282787322998 disk.rbd3.out‚ 1.36328125 network.recv‚553235 disk.rbd2.in‚0 proc.cpu.threads‚17 disk.sda3.in‚0.109375 disk.dm-1.out‚ 61.80859375! disk.rbd2.out‚118.15283203125 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.dm-0.in‚ 64.4921875 disk.sda2.out‚0“ŒÙ ‚(:’ çÚâÔå¹u gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚134.998' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.999599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ö+Z:Ú ‚(:¯ çÚâÔž½‰x network.recv‚558245) disk./.usagePercent‚70.07015609987718% disk.nvme0n1.out‚8722.00732421875 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd3.out‚ 1.36328125 disk.nvme0n1.in‚ 64.4921875 disk.sda2.out‚0$ memory_percent‚86.92760467529297! proc.memory.rssMB‚ 2910.265625 network.sent‚ 3.107729e+06 disk.sda2.in‚0) proc.memory.percent‚17.76285171508789" disk.dm-0.out‚8722.00732421875 disk.dm-1.out‚ 62.296875 disk.rbd1.in‚0 disk.sda1.in‚0 disk.dm-0.in‚ 64.4921875 disk.sda3.out‚ 62.296875 disk.rbd2.in‚0 disk.rbd0.out‚0.078125 disk.dm-1.in‚0.109375' disk.nvme0c0n1.out‚8722.00732421875& proc.memory.availableMB‚ 2141.78125! disk.rbd2.out‚118.15673828125 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.nvme0c0n1.in‚ 64.4921875$ disk./.usageGB‚1044.095962524414 disk.rbd1.out‚ 0.982421875 disk.sda.out‚ 62.296875 disk.rbd3.in‚0 disk.sda3.in‚0.109375 cpu‚24.725331478962296 proc.cpu.threads‚17òê2Û - _wandbruntime‚1525  _runtime‚15257ñõÆÜ À & lr-AdamW/pg2‚0.0015521968062439993 trainer/global_step‚1699 $ _timestamp‚1.788390769739348e+09 _runtime‚1525.495311025 _step‚134 & lr-AdamW/pg1‚0.0015521968062439993ÓPÛHËÝ Å _step‚134 & lr-AdamW/pg1‚0.0015521968062439993 & lr-AdamW/pg2‚0.0015521968062439993 trainer/global_step‚1699 $ _timestamp‚1.788390769739348e+09 _runtime‚1525.495311025† kAÂ2Þ - _wandbruntime‚1525  _runtime‚1525iU Xâ'ß ÜO TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36103799 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚114845648939520 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _step‚135 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36081202 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17661384 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28882968 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚115808052234752 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9028055 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚115807791496192 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚110759544080384 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64964170 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64986769 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚110759561119744 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1382766967545 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚113455080224768 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28882968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚36103799 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1382574971009 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚113455097264128 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚553 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738560 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36081202 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚556 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚114352187591168 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114845435721728 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚113455080224768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1390551675392 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1390355496960ЂQHç'à áO L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28882968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚36103799 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1382574971009 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚113455097264128 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚553 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738560 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36081202 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚556 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚114352187591168 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114845435721728 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚113455080224768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1390551675392 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1390355496960 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36103799 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚114845648939520 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _step‚135 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36081202 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17661384 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28882968 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚115808052234752 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9028055 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚115807791496192 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚110759544080384 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64964170 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64986769 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚110759561119744 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1382766967545 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚113455080224768‡_¹Ý=2á - _wandbruntime‚1526  _runtime‚1526)%Ü°Ä%â æM ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚114417688183808 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚112206721355021 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36124697 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1391153887744 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚114417713087488 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚514 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚114911486380544 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1456473878528 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28899488 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚115874162062336 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28899490 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚110823143464960 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36124697 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1391350079488 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚65001562 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28899488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚114911273149440 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚113520119261696 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚65024187 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9033531 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚136 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17671421 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9033020 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17671418 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36102074 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1383368853957 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26704438 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚115874431176192 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚65001562 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1456718088704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112206512318917 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1383560850701 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36102074 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113520119261696 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244210176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28899490 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚113520136301056 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚1699 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚114911486380544 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113856 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 % _timestamp‚1.7883907705083432e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚110823160504320 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26704952 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744{³8Æñ&ã ëM NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28899490 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚113520136301056 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚1699 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚114911486380544 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113856 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 % _timestamp‚1.7883907705083432e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚110823160504320 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26704952 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚114417688183808 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚112206721355021 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36124697 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1391153887744 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚114417713087488 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚514 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚114911486380544 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1456473878528 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28899488 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚115874162062336 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28899490 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚110823143464960 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36124697 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1391350079488 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚65001562 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28899488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚114911273149440 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚113520119261696 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚65024187 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9033531 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚136 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17671421 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9033020 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17671418 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36102074 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1383368853957 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26704438 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚115874431176192 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚65001562 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1456718088704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112206512318917 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1383560850701 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36102074 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113520119261696 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244210176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360ˆà vâ2ä - _wandbruntime‚1526  _runtime‚1526¯þZXå Ò 0loss/FBt4/assignment_loss‚0.2709001309054796 _runtime‚ 1526.26609517 0loss/FBt1/assignment_loss‚0.2092192866563748 1loss/FBt2/assignment_loss‚0.11259788316688638 /loss/FRt4/detection_loss‚2.4725390061354027 1loss/SRqqt1/detection_loss‚1.7809491384530736 /loss/FRt1/assignment_loss‚3.710372908429785 2loss/SRqqt1/assignment_loss‚1.8116420584025783 /loss/FBt2/detection_loss‚0.6906375455599325 % _timestamp‚1.7883907705103214e+09 .loss/FRt1/detection_loss‚2.387435977930551 1loss/SRqqt4/detection_loss‚2.5393099317948025 %loss/total_loss‚47.23965727592882 0loss/FRt2/assignment_loss‚2.0197404999177433 .loss/FRt3/assignment_loss‚3.66963419807491 /loss/FRt4/assignment_loss‚4.098715410782741 2loss/SRqqt2/assignment_loss‚1.4904132182044643 1loss/SRqqt2/detection_loss‚1.1163194919271129 epoch‚2 2loss/SRqqt3/assignment_loss‚1.6792249294864658 1loss/SRqqt4/assignment_loss‚2.042646885663271 .loss/FRt2/detection_loss‚1.791899385165821 /loss/FRt3/detection_loss‚2.7003013905304583 1loss/SRqqt3/detection_loss‚1.8824861096863699 /loss/FBt1/detection_loss‚2.4534440573893095 trainer/global_step‚1699 1loss/FBt3/assignment_loss‚0.11859471672214568 .loss/FBt3/detection_loss‚3.003086394071579 /loss/FBt4/detection_loss‚3.1875467208715587 _step‚137÷[I]æ × .loss/FRt1/detection_loss‚2.387435977930551 1loss/SRqqt4/detection_loss‚2.5393099317948025 %loss/total_loss‚47.23965727592882 0loss/FRt2/assignment_loss‚2.0197404999177433 .loss/FRt3/assignment_loss‚3.66963419807491 /loss/FRt4/assignment_loss‚4.098715410782741 2loss/SRqqt2/assignment_loss‚1.4904132182044643 1loss/SRqqt2/detection_loss‚1.1163194919271129 epoch‚2 2loss/SRqqt3/assignment_loss‚1.6792249294864658 1loss/SRqqt4/assignment_loss‚2.042646885663271 .loss/FRt2/detection_loss‚1.791899385165821 /loss/FRt3/detection_loss‚2.7003013905304583 1loss/SRqqt3/detection_loss‚1.8824861096863699 /loss/FBt1/detection_loss‚2.4534440573893095 trainer/global_step‚1699 1loss/FBt3/assignment_loss‚0.11859471672214568 .loss/FBt3/detection_loss‚3.003086394071579 /loss/FBt4/detection_loss‚3.1875467208715587 _step‚137 0loss/FBt4/assignment_loss‚0.2709001309054796 _runtime‚ 1526.26609517 0loss/FBt1/assignment_loss‚0.2092192866563748 1loss/FBt2/assignment_loss‚0.11259788316688638 /loss/FRt4/detection_loss‚2.4725390061354027 1loss/SRqqt1/detection_loss‚1.7809491384530736 /loss/FRt1/assignment_loss‚3.710372908429785 2loss/SRqqt1/assignment_loss‚1.8116420584025783 /loss/FBt2/detection_loss‚0.6906375455599325 % _timestamp‚1.7883907705103214e+09‰ß%œç ‚(:‘ öÚâÔžªµu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.288' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.257600000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0• Þ¯Cè ‚(:¸ öÚâÔÀäãw! disk.nvme0c0n1.in‚ 64.49609375 disk.sda2.out‚0 disk.dm-1.in‚0.109375! proc.memory.rssMB‚ 2910.265625' disk.nvme0c0n1.out‚8722.86474609375 disk.rbd3.out‚ 1.36328125" disk.dm-0.out‚8722.86474609375 disk.sda3.in‚0.109375 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.rbd3.in‚0) proc.memory.percent‚17.76285171508789 cpu‚24.734133037084842 disk.sda.out‚ 62.8046875 disk.sda1.out‚0 disk.nvme0n1.in‚ 64.49609375$ memory_percent‚86.92927360534668 disk.drbd1777.in‚0 disk.dm-0.in‚ 64.49609375 disk.sda3.out‚ 62.8046875 disk.rbd1.out‚ 0.982421875( proc.memory.availableMB‚ 2141.5078125 proc.cpu.threads‚17 network.sent‚ 3.113005e+06 disk.sda.in‚0.109375 disk.rbd2.in‚0! disk.rbd2.out‚129.75048828125% disk.nvme0n1.out‚8722.86474609375 network.recv‚562034% disk./.usageGB‚1044.0962600708008 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.dm-1.out‚ 62.8046875 disk.rbd1.in‚0) disk./.usagePercent‚70.07017606846485~Tëœé ‚(:‘ …ÛâÔ†ê¨u gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.439' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.087799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ˆCýCê ‚(:¸ …ÛâÔ¯šŽx disk.rbd3.out‚ 1.36328125 disk.dm-1.in‚0.109375$ disk.nvme0n1.out‚8821.4775390625 disk.sda.out‚ 63.30859375 disk.sda1.out‚0% disk./.usageGB‚1044.0962562561035 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd1.in‚0 disk.rbd0.in‚0$ memory_percent‚86.92934513092041 disk.sda1.in‚0 disk.sda2.in‚0 network.sent‚ 3.199671e+06! proc.memory.rssMB‚ 2910.265625 cpu‚24.74230022724653 disk.rbd3.in‚0 disk.rbd1.out‚ 0.982421875 disk.rbd0.out‚0.078125) proc.memory.percent‚17.76285171508789 disk.dm-0.in‚ 64.49609375& disk.nvme0c0n1.out‚8821.4775390625 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.sda3.in‚0.109375! disk.rbd2.out‚129.75048828125) proc.memory.availableMB‚ 2141.49609375 proc.cpu.threads‚17 disk.dm-1.out‚ 63.30859375 disk.nvme0n1.in‚ 64.49609375! disk.dm-0.out‚8821.4775390625 network.recv‚567540) disk./.usagePercent‚70.07017581245731 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 64.49609375 disk.sda3.out‚ 63.30859375‚Ë\’ë ‚(:‡ ”ÛâÔǃ­w gpu.0.gpu‚61.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.842' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.7684 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0à‡ g/ì ‚(:¤ ”ÛâÔ¿“áy network.recv‚572045 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.in‚0.109375 disk.sda.in‚0.109375 disk.sda.out‚63.75% disk./.usageGB‚1044.0973854064941 disk.drbd1777.out‚0 disk.rbd1.in‚0) proc.memory.availableMB‚ 2141.48828125 proc.cpu.threads‚17# disk.nvme0n1.out‚8920.873046875$ memory_percent‚86.92939281463623) disk./.usagePercent‚70.07025159068745 disk.dm-0.in‚ 64.49609375 disk.sda3.out‚63.75 disk.dm-1.in‚0.109375 disk.dm-1.out‚63.75 network.sent‚ 3.205788e+06 disk.sda2.in‚0 disk.rbd1.out‚ 0.982421875 disk.rbd2.in‚0! disk.rbd2.out‚131.54736328125 disk.sda1.out‚0 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 64.49609375% disk.nvme0c0n1.out‚8920.873046875 disk.rbd3.in‚0) proc.memory.percent‚17.76285171508789 disk.drbd1777.in‚0 disk.dm-0.out‚8920.873046875 disk.nvme0n1.in‚ 64.49609375! proc.memory.rssMB‚ 2910.265625 cpu‚24.750730389278452¡¨<¨2í - _wandbruntime‚1567  _runtime‚1567¸pÌ^Äî ¾ trainer/global_step‚1749 $ _timestamp‚1.788390811998903e+09 _runtime‚1567.754920281 _step‚138 % lr-AdamW/pg1‚0.001537224260007589 % lr-AdamW/pg2‚0.0015372242600075896[SÉï à % lr-AdamW/pg1‚0.001537224260007589 % lr-AdamW/pg2‚0.001537224260007589 trainer/global_step‚1749 $ _timestamp‚1.788390811998903e+09 _runtime‚1567.754920281 _step‚138Š^€ÏN2ð - _wandbruntime‚1567  _runtime‚1567CD?â'ñ ÜO ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118137548153344 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚536 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118137334935552 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚117628437503488 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37128154 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚1567.760003038 _step‚139 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18163084 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚117628462407168 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29708970 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚116707049110528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1430302864384 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118137334935552 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27451900 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚116707049110528 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66859721 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚119126463120896 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚119126196090880 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29708970 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚539 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1430499042816 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚115362018635497 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37128154 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚115361809599601 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1422296290417 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27451361 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113939530348544 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113939513309184 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚116707032071168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29708968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚37150751 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66837122 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚1749 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚37150751 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1430499042816 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1422488286953 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1497758587392 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 % _timestamp‚1.7883908119999747e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118137548153344 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18163087 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9288813 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29708968 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚116707032071168 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030016 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750¡Ÿ#¼7ò áO eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚117628462407168 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29708970 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚116707049110528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1430302864384 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118137334935552 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27451900 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚116707049110528 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66859721 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚119126463120896 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚119126196090880 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29708970 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚539 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1430499042816 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚115362018635497 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37128154 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚115361809599601 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1422296290417 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27451361 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113939530348544 F@DeviceStatsMonitor.on_train_batch_start/segm0×M,°ent.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113939513309184 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚116707032071168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27580 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29708968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚37150751 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66837122 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚1749 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚37150751 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1430499042816 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1422488286953 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1497758587392 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 % _timestamp‚1.7883908119999747e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118137548153344 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18163087 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9288813 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29708968 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚116707032071168 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030016 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118137548153344 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚536 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118137334935552 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚117628437503488 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37128154 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚1567.760003038 _step‚139 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18163084‹ššvè2ó - _wandbruntime‚1568  _runtime‚1568‡ýÐì&ô æM E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883908127635214e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚118203385594368 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37171649 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66874514 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1423282170109 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114003112693760 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _step‚140 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118203172363264 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66897139 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27466516 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233724416 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1431101255168 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚116772088147456 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1431297446912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66874514 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚114003129733120 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27467054 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18173118 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29725488 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1431101255168 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚119192572096512 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37171649 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37149026 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚117693987903488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258628096 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29725488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚115426411903229 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚116772071108096 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66897139 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚117693962999808 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _runtime‚1568.520071307 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18173121 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1498842821120 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1423090173365 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1423090173365 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37171649 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66874514 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1423282170109 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114003112693760 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _step‚140 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690048 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534400 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118203172363264 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66897139 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27466516 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233724416 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1431101255168 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚116772088147456 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1431297446912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66874514 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚114003129733120 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27467054 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18173118 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29725488 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1431101255168 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22623 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚119192572096512 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37171649 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37149026 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚117693987903488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231104 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258628096 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196191744 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29725488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚115426411903229 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚116772071108096 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66897139 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚117693962999808 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _runtime‚1568.520071307ŒÝ72ö - _wandbruntime‚1568  _runtime‚1568ÝDÒmY÷ Ó epoch‚2 _runtime‚1568.521160921 2loss/SRqqt1/assignment_loss‚1.9309110713750124 1loss/SRqqt3/detection_loss‚1.9106704738736153 % _timestamp‚1.7883908127653587e+09 /loss/FRt3/assignment_loss‚3.255143433411916 0loss/SRqqt4/assignment_loss‚2.09636127948761 0loss/FBt1/assignment_loss‚0.3090303912758827 .loss/FRt4/detection_loss‚2.679164352826774 /loss/FBt3/detection_loss‚1.9155174642801285 trainer/global_step‚1749 1loss/FBt4/assignment_loss‚0.41896797862427776 .loss/FRt1/detection_loss‚2.468150471958021 /loss/FRt2/detection_loss‚1.6585319673913157 1loss/SRqqt4/detection_loss‚2.6170896170660853 _step‚141 /loss/FRt3/detection_loss‚2.5923036670684816 1loss/SRqqt2/detection_loss‚1.0688981325735176 /loss/FRt4/assignment_loss‚4.310239441692829 1loss/FBt3/assignment_loss‚0.39665982181904835 .loss/FBt2/detection_loss‚0.649832520838944 /loss/FRt1/assignment_loss‚3.503862344349424 &loss/total_loss‚46.675311839322674 2loss/SRqqt2/assignment_loss‚1.4414060318931228 0loss/FBt2/assignment_loss‚0.1423830796052086 /loss/FBt1/detection_loss‚2.7015891353289287 /loss/FRt2/assignment_loss‚2.364258184392228 2loss/SRqqt3/assignment_loss‚1.6562499276548623 1loss/SRqqt1/detection_loss‚1.8916887809336185 /loss/FBt4/detection_loss‚2.6964022696018217˜÷»©^ø Ø epoch‚2 _runtime‚1568.521160921 2loss/SRqqt1/assignment_loss‚1.9309110713750124 1loss/SRqqt3/detection_loss‚1.9106704738736153 % _timestamp‚1.7883908127653587e+09 /loss/FRt3/assignment_loss‚3.255143433411916 0loss/SRqqt4/assignment_loss‚2.09636127948761 0loss/FBt1/assignment_loss‚0.3090303912758827 .loss/FRt4/detection_loss‚2.679164352826774 /loss/FBt3/detection_loss‚1.9155174642801285 trainer/global_step‚1749 1loss/FBt4/assignment_loss‚0.41896797862427776 .loss/FRt1/detection_loss‚2.468150471958021 /loss/FRt2/detection_loss‚1.6585319673913157 1loss/SRqqt4/detection_loss‚2.6170896170660853 _step‚141 /loss/FRt3/detection_loss‚2.5923036670684816 1loss/SRqqt2/detection_loss‚1.0688981325735176 /loss/FRt4/assignment_loss‚4.310239441692829 1loss/FBt3/assignment_loss‚0.39665982181904835 .loss/FBt2/detection_loss‚0.649832520838944 /loss/FRt1/assignment_loss‚3.503862344349424 &loss/total_loss‚46.675311839322674 2loss/SRqqt2/assignment_loss‚1.4414060318931228 0loss/FBt2/assignment_loss‚0.1423830796052086 /loss/FBt1/detection_loss‚2.7015891353289287 /loss/FRt2/assignment_loss‚2.364258184392228 2loss/SRqqt3/assignment_loss‚1.6562499276548623 1loss/SRqqt1/detection_loss‚1.8916887809336185 /loss/FBt4/detection_loss‚2.6964022696018217Ί˜{ù ‚(:’ £ÛâÔÆðÃu gpu.0.gpu‚23.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚155.216' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.043200000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 U|Cú ‚(:¸ £ÛâÔÅÑÀx disk.dm-0.in‚ 64.49609375 disk.sda1.in‚0 disk.sda2.in‚0 proc.cpu.threads‚17 disk.sda.in‚0.109375) proc.memory.percent‚17.76285171508789 disk.sda3.in‚0.109375 disk.drbd1777.in‚0! proc.memory.rssMB‚ 2910.265625 disk.dm-1.out‚ 64.33984375! disk.rbd2.out‚131.55126953125 disk.nvme0n1.in‚ 64.49609375( proc.memory.availableMB‚ 2141.7109375 disk.rbd3.in‚0! disk.dm-0.out‚8929.3505859375 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875 disk.sda.out‚ 64.33984375 disk.rbd0.in‚0$ memory_percent‚86.92803382873535 network.recv‚576687) disk./.usagePercent‚70.07071368428676 disk.sda3.out‚ 64.33984375 disk.sda1.out‚0& disk.nvme0c0n1.out‚8929.3505859375% disk./.usageGB‚1044.1042709350586 disk.rbd3.out‚ 1.36328125 disk.dm-1.in‚0.109375 disk.rbd2.in‚0$ disk.nvme0n1.out‚8929.3505859375 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 64.49609375 cpu‚24.758801113944497 network.sent‚ 3.212381e+06 t}œû ‚(:‘ ²ÛâÔ’êÒu gpu.0.gpu‚33.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.083' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.816599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0•KP>ü ‚(:³ ²ÛâÔ†©Çx disk.sda3.out‚ 65.734375 disk.dm-1.in‚0.109375! disk.rbd2.out‚131.61376953125 disk.nvme0n1.in‚ 64.49609375 disk.sda.in‚0.109375 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 64.49609375& disk.nvme0c0n1.out‚9028.9326171875 network.sent‚ 3.298992e+06) disk./.usagePercent‚70.07072162052032 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚ 64.49609375) proc.memory.percent‚17.76285171508789 cpu‚24.766688883754576$ disk.nvme0n1.out‚9028.9326171875 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.drbd1777.out‚0 disk.sda2.in‚0$ memory_percent‚86.92820072174072 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.sda.out‚ 65.734375 disk.sda1.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 0.982421875 network.recv‚582456% disk./.usageGB‚1044.1043891906738! disk.dm-0.out‚9028.9326171875 disk.dm-1.out‚ 65.734375) proc.memory.availableMB‚ 2141.68359375! proc.memory.rssMB‚ 2910.265625Ã,×’ý ‚(:‡ ÁÛâԲݤu gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.197' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.2394 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Õ‘±þ ‚(:¸ ÁÛâÔÒ¤ýx& disk.nvme0c0n1.out‚9128.2763671875) proc.memory.availableMB‚ 2141.42578125) proc.memory.percent‚17.76285171508789 disk.dm-0.in‚ 64.49609375 disk.sda1.out‚0 disk.drbd1777.in‚0! proc.memory.rssMB‚ 2910.265625$ disk.nvme0n1.out‚9128.2763671875 disk.rbd1.in‚0! disk.dm-0.out‚9128.2763671875 disk.sda3.out‚ 66.28515625 disk.nvme0n1.in‚ 64.49609375 proc.cpu.threads‚17 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 64.49609375# memory_percent‚86.9297742843628 cpu‚24.774134809067046 disk.dm-1.out‚ 66.28515625 disk.sda3.in‚0.109375 disk.sda2.out‚0 disk.sda.o&ûù ’ut‚ 66.28515625 disk.sda1.in‚0 disk.rbd1.out‚ 0.982421875) disk./.usagePercent‚70.07078383035113% disk./.usageGB‚1044.1053161621094 disk.rbd3.out‚ 1.36328125 network.sent‚ 3.303436e+06 network.recv‚585069! disk.rbd2.out‚131.61376953125 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 ú—2ÿ - _wandbruntime‚1610  _runtime‚1610X#}Ç€ Á & lr-AdamW/pg1‚0.0015216856901994985 & lr-AdamW/pg2‚0.0015216856901994985 trainer/global_step‚1799 % _timestamp‚1.7883908543036249e+09 _runtime‚1610.059589794 _step‚142ªA×BÌ Æ % _timestamp‚1.7883908543036249e+09 _runtime‚1610.059589794 _step‚142 & lr-AdamW/pg1‚0.0015216856901994985 & lr-AdamW/pg2‚0.0015216856901994985 trainer/global_step‚1799Ž¢Ú©2‚ - _wandbruntime‚1610  _runtime‚1610!e_â'ƒ ÜO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1470446410240 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883908543046334e+09 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚119959000956928 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚121429234149376 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚121429234149376 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1462209606361 _step‚143 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1470446410240 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30534968 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38175106 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚119958983917568 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68732673 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚68710074 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30534970 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚119959000956928 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 trainer/global_step‚1799 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚121429447367168 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚122444916233216 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18664787 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚38197703 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1539936883712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚122444649203200 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚30534968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚119958983917568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚120904737223168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚117119482537984 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1470250231808 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1470250231808 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28212457 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18664784 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9547169 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030016 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚118581500147809 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚1610.064027295 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38175106 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚120904712319488 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚117119499577344 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28211953 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1540179010048 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1462017609825 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0¦/Óç'„ áO G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚121429234149376 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22599 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27327 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1462209606361 _step‚143 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1470446410240 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30534968 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38175106 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534400 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚119958983917568 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68732673 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996536 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚68710074 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30534970 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚119959000956928 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690048 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 trainer/global_step‚1799 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚121429447367168 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213217792 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚122444916233216 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035896 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534400 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18664787 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213217792 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚38197703 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1539936883712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚122444649203200 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640709 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚30534968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚119958983917568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22597 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚120904737223168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚117119482537984 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1470250231808 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690048 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1470250231808 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28212457 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18664784 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9547169 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267030016 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚118581500147809 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27580 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚1610.064027295 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242126336 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22597 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38175106 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚120904712319488 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚117119499577344 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27327 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178432 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28211953 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1540179010048 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1462017609825 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22599 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1470446410240 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 % _timestamp‚1.7883908543046334e+09 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚119959000956928 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚121429234149376 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 IDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚120024039993856 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1540760908288 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚118646102451437 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚121495284808192 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚120024022954496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚122510998724096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 trainer/global_step‚1799 % _timestamp‚1.7883908550580935e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38195978 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚68747466 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38195978 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚122511276226560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30551488 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68770091 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚120970262719488 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚121495071577088 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1471244814336 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚68747466 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1462811492773 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18674818 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1471244814336 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18674821 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38218601 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1471048622592 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1471048622592 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚68770091 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚118645893415333 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1541013507072 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28227061 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9552243 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚519 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277502464 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚1610.814470719 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚118646102451437 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚121495284808192 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690048 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚120024022954496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196191744 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27580 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27327 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚122510998724096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996744 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22623 trainer/global_step‚1799 % _timestamp‚1.7883908550580935e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22625 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38195978 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚68747466 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38195978 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚122511276226560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036104 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30551488 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68770091 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚120970262719488 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚121495071577088 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1471244814336 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚68747466 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1462811492773 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18674818 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1471244814336 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18674821 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27580 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38218601 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1471048622592 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1471048622592 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚68770091 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚118645893415333 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1541013507072 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28227061 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231104 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9552243 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚519 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277502464 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640709 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚1610.814470719 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚120024039993856 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1540760908288 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0Û(üÍ2ˆ - _wandbruntime‚1610  _runtime‚1610 ‡"}V‰ Ð 2loss/SRqqt2/assignment_loss‚1.4102346967876882 0loss/FBt3/assignment_loss‚0.4488284165039659 -loss/FRt3/detection_loss‚2.45036866573187 .loss/FRt4/detection_loss‚2.841565320124993 1loss/SRqqt3/detection_loss‚1.9524758433040819 /loss/FBt1/detection_loss‚2.6074151851628957 /loss/FBt4/detection_loss‚2.7777222260832786 0loss/FRt1/assignment_loss‚3.7499577273493228 0loss/FRt2/assignment_loss‚2.1157052445411684 1loss/FBt2/assignment_loss‚0.15501406950428112 /loss/SRqqt1/detection_loss‚1.85114053791424 /loss/FBt3/detection_loss‚2.2940230637788774 %loss/total_loss‚47.67967076152367 /loss/FRt4/assignment_loss‚4.399020681014428 1loss/SRqqt4/assignment_loss‚2.048194107306855 1loss/FBt1/assignment_loss‚0.33457660466726674 trainer/global_step‚1799 _runtime‚1610.815650733 _step‚145 /loss/FBt2/detection_loss‚0.4931932617207812 $ _timestamp‚1.788390855059851e+09 /loss/FRt3/assignment_loss‚3.842660969430274 2loss/SRqqt3/assignment_loss‚2.0678758718465504 1loss/SRqqt2/detection_loss‚1.0131949270649474 0loss/SRqqt1/assignment_loss‚1.85678997045418 1loss/FBt4/assignment_loss‚0.29047259415383453 /loss/FRt1/detection_loss‚2.4489882611062215 /loss/FRt2/detection_loss‚1.8092797157168388 0loss/SRqqt4/detection_loss‚2.420972800254822 epoch‚2èÅC[Š Õ /loss/FBt4/detection_loss‚2.7777222260832786 0loss/FRt1/assignment_loss‚3.7499577273493228 0loss/FRt2/assignment_loss‚2.1157052445411684 1loss/FBt2/assignment_loss‚0.15501406950428112 /loss/SRqqt1/detection_loss‚1.85114053791424 /loss/FBt3/detection_loss‚2.2940230637788774 %loss/total_loss‚47.67967076152367 /loss/FRt4/assignment_loss‚4.399020681014428 1loss/SRqqt4/assignment_loss‚2.048194107306855 1loss/FBt1/assignment_loss‚0.33457660466726674 trainer/global_step‚1799 _runtime‚1610.815650733 _step‚145 /loss/FBt2/detection_loss‚0.4931932617207812 $ _timestamp‚1.788390855059851e+09 /loss/FRt3/assignment_loss‚3.842660969430274 2loss/SRqqt3/assignment_loss‚2.0678758718465504 1loss/SRqqt2/detection_loss‚1.0131949270649474 0loss/SRqqt1/assignment_loss‚1.85678997045418 1loss/FBt4/assignment_loss‚0.29047259415383453 /loss/FRt1/detection_loss‚2.4489882611062215 /loss/FRt2/detection_loss‚1.8092797157168388 0loss/SRqqt4/detection_loss‚2.420972800254822 epoch‚2 2loss/SRqqt2/assignment_loss‚1.4102346967876882 0loss/FBt3/assignment_loss‚0.4488284165039659 -loss/FRt3/detection_loss‚2.45036866573187 .loss/FRt4/detection_loss‚2.841565320124993 1loss/SRqqt3/detection_loss‚1.9524758433040819 /loss/FBt1/detection_loss‚2.6074151851628957‘ §ûÆŽ‹ ‚(:ƒ ÐÛâÔ¡ñÞu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.6' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.52 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢«H BŒ ‚(:· ÐÛâÔñ¸¨x disk.dm-0.in‚ 64.49609375 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0! proc.memory.rssMB‚ 2910.265625 network.sent‚ 3.30959e+06 disk.rbd0.in‚0 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 64.49609375 disk.rbd3.in‚0( proc.memory.availableMB‚ 2141.3671875 network.recv‚589706) disk./.usagePercent‚70.07079099856209 disk.sda2.out‚0" disk.dm-0.out‚9130.35302734375 disk.rbd2.in‚0% disk.nvme0n1.out‚9130.35302734375 disk.sda.in‚0.109375 disk.sda.out‚ 66.5546875% disk./.usageGB‚1044.1054229736328 disk.dm-1.in‚0.109375 disk.rbd1.out‚ 0.982421875 disk.sda1.out‚0$ memory_percent‚86.93013191223145) proc.memory.percent‚17.76285171508789' disk.nvme0c0n1.out‚9130.35302734375 disk.sda3.in‚0.109375 disk.dm-1.out‚ 66.5546875 disk.rbd1.in‚0! disk.rbd2.out‚132.87548828125 disk.nvme0n1.in‚ 64.49609375 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚ 66.5546875 cpu‚24.781773521441927 proc.cpu.threads‚17OÀõœ ‚(:‘ ßÛâÔÄþu gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚110.286' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.057199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:›“>Ž ‚(:³ ßÛâÔò‘¸x) proc.memory.percent‚17.76285171508789 network.sent‚ 3.397343e+06' disk.nvme0c0n1.out‚9228.80908203125 disk.sda2.in‚0' proc.memory.availableMB‚ 2141.390625 network.recv‚595919 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.sda.out‚ 67.265625$ memory_percent‚86.92998886108398 proc.cpu.threads‚17 disk.rbd0.out‚0.078125 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd1.out‚ 0.982421875! proc.memory.rssMB‚ 2910.265625) disk./.usagePercent‚70.07107542293264 disk.nvme0n1.in‚ 64.49609375 disk.sda1.in‚0 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 64.49609375 disk.sda2.out‚0 disk.dm-0.in‚ 64.49609375 cpu‚24.842149495237628! disk.rbd2.out‚132.90673828125 disk.drbd1777.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-1.out‚ 67.265625$ disk./.usageGB‚1044.109661102295% disk.nvme0n1.out‚9228.80908203125" disk.dm-0.out‚9228.80908203125 disk.sda3.out‚ 67.265625 disk.rbd1.in‚0ªtªÁ‘ ‚(:† îÛâÔ§ Æu gpu.0.gpu‚70.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚88.876' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7752 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0núyC ‚(:’ îÛâÔ²µžx network.sent‚ 3.402661e+06) disk./.usagePercent‚70.07103983788538 disk.dm-0.in‚64.5 disk.sda3.in‚0.109375% disk.nvme0n1.out‚9329.84326171875 disk.rbd0.out‚0.078125 proc.memory.rssMB‚ 2920.46875' disk.nvme0c0n1.out‚9329.84326171875 disk.sda2.in‚0 disk.sda.in‚0.109375 disk.sda1.in‚0 disk.sda3.out‚67.75 disk.rbd1.out‚ 0.982421875! disk.rbd2.out‚134.13720703125 disk.sda.out‚67.75 network.recv‚599618 disk.sda1.out‚0 cpu‚24.900790466802352$ disk./.usageGB‚1044.109130859375 disk.rbd3.in‚0" disk.dm-0.out‚9329.84326171875 disk.dm-1.out‚67.75 disk.rbd1.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0( proc.memory.availableMB‚ 2131.3984375 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.nvme0n1.in‚64.5 disk.drbd1777.out‚0$ memory_percent‚86.99097633361816) proc.memory.percent‚17.82512664794922 proc.cpu.threads‚17 disk.nvme0c0n1.in‚64.5 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125”±šy2‘ - _wandbruntime‚1662  _runtime‚1662‡÷Û˜A’ õ§ 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4019857943058014 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8888888955116272 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3524305522441864 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.22304147481918335 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2848106026649475 8"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.308824747800827 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3405461013317108 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3387680947780609 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6000000238418579 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.1679399311542511 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9969950318336487 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3873119652271271 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3068452477455139 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9978922605514526 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3333333134651184 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 (particle/f_score‚0.37799370288848877 particle/accuracy_1_of_6‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.35740038752555847 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9"Purity/0FRt2SRqqt1FBt/event_purity‚0.0197132620960474 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.10879630595445633 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ) Purity/0FRt*SRqqt4FBt/FBt_purity‚0.75 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3206731379032135 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.16128724813461304 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24217171967029572 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.038148149847984314 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9473055601119995 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 6 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.800000011920929 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3142682909965515 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.981205403804779 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8818008899688721 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.09114398807287216 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 particle/accuracy_3_of_3‚1 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1257437914609909 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.28553009033203125 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.13106638193130493 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3354838490486145 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2719298303127289 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1944444179534912 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.005063365679234266 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9867998361587524 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27742448449134827 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17851795256137848 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.21581311523914337 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2716553509235382 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9858120083808899 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2514374256134033 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.30906835198402405 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2914920747280121 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3333333134651184 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9933744668960571 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.30399808287620544 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.06871610134840012 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33166345953941345 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15756537020206451 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.8855159878730774 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.42927730083465576 6 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.147939071059227 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9837575554847717 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.31585487723350525 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9855555295944214 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16803213953971863 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11499813199043274 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9765745401382446 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.998068630695343 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.29101210832595825 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.04597701132297516 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.048966627568006516 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08291006088256836 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9736841917037964 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 _runtime‚1662.152945186 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.7916666865348816 particle/accuracy_2_of_6‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08973724395036697 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_5‚1 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.30577200651168823 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2887383699417114 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5333333611488342 ,jet/accuracy_4_of_5‚0.023156682029366493 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3749999701976776 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3628650903701782 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 particle/accuracy_4_of_6‚1 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3946434259414673 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11730237305164337 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.11806695908308029 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_3‚0.01566033437848091 particle/accuracy_5_of_5‚1 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1525263488292694 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.719298243522644 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.32894736528396606 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.25132274627685547 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.19469699263572693 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 6 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.069444440305233 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.1988518387079239 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 8"Purity/1FRt0SRqqt*FBt/event_purity‚0.123995341360569 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10711078345775604 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.36249998211860657 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9892921447753906 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22433899343013763 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.07894736528396606 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.44771161675453186 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3033333122730255 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.3125 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8621399402618408 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9620370268821716 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00435002101585269 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.36385229229927063 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_7‚0.5 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07645847648382187 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 6 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.980008602142334 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.41939249634742737 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9873471260070801 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3054475784301758 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2866467237472534 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.04938271641731262 6  ª‹ËdRPurity/1FRt1SRqqt*FBt/FRt_purity‚0.161896750330925 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11005154997110367 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_7‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3809758424758911 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3129967153072357 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.8755555748939514 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1291532665491104 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.0882352963089943 particle/accuracy_1_of_1‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.975712239742279 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01904761977493763 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9948257803916931 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30687594413757324 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666746139526 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3484416902065277 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03953506425023079 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16807568073272705 8"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.324735552072525 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.08755991607904434 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 jet/accuracy_2_of_7‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24703863263130188 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.34129753708839417 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.41111108660697937 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 jet/accuracy_7_of_7‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9672619104385376 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 +jet/accuracy_2_of_3‚0.18701666593551636 jet/accuracy_4_of_6‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.27869048714637756 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43358293175697327 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9970680475234985 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1096554845571518 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3364197909832001 (particle/accuracy‚0.8461232781410217 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4261505901813507 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.36666667461395264 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.028148148208856583 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 jet/accuracy_5_of_5‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3261164426803589 particle/accuracy_4_of_5‚1 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0714285746216774 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04158828780055046 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014995631761848927 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7017543911933899 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.1666666716337204 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03424396738409996 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.33725306391716003 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 8"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.499909907579422 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.4035087823867798 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4117456078529358 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.10869566351175308 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 _step‚146 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9974899291992188 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.09114398807287216 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.16572986543178558 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2666539251804352 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3898809552192688 ,jet/accuracy_3_of_4‚0.047357045114040375 particle/accuracy_2_of_7‚1 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3384050726890564 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3255474865436554 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9867740273475647 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.019095269963145256 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.47279250621795654 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.13183583319187164 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9830211997032166 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14694549143314362 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9838218092918396 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_1_of_5‚1 6validation_average_jet_accuracy‚0.3614272475242615 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.39734262228012085 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3333333134651184 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16939488053321838 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.09824734181165695 jet/accuracy_6_of_6‚0 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34838438034057617 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.26777610182762146 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 *loss/val_total_loss‚48.059564018326675 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2806102931499481 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.013470638543367386 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.20370371639728546 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0008488964522257447 particle/accuracy_6_of_6‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15923736989498138 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6333677172660828 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.008824329823255539 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3242565393447876 particle/accuracy_3_of_4‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03536558151245117 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9931899905204773 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.33081796765327454 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9789924621582031 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2718239724636078 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9978922605514526 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 jet/accuracy_4_of_7‚0 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.37073731422424316 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.061972636729478836 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 *jet/accuracy_1_of_4‚0.8009987473487854 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9987633228302002 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3844726085662842 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.47066324949264526 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.13579638302326202 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.33888888359069824 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8063710927963257 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.43203699588775635 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 particle/accuracy_1_of_4‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_4‚0.27843645215034485 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9958208203315735 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.17123809456825256 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.18312756717205048 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3359375 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31077101826667786 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 +jet/accuracy_1_of_1‚0.39242878556251526 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9966565370559692 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16261513531208038 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9908744096755981 6 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.272583931684494 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.14294248819351196 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_2‚0.6412614583969116 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3316250741481781 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2956826090812683 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ) Purity/0FRt0SRqqt4FBt/FBt_purity‚0.75 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3584555387496948 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014283781871199608 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 trainer/global_step‚1811 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3177418112754822 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9644259810447693 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3976934552192688 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10465293377637863 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_1_of_7‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.028033796697854996 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 ,particle/sensitivity‚0.29949626326560974 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6706352233886719 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.07999999821186066 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.21785713732242584 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.34598925709724426 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0026881720405071974 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09400128573179245 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9267126321792603 jet/accuracy_5_of_7‚0 particle/accuracy_4_of_7‚1 % _timestamp‚1.7883909063952086e+09 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.15889158844947815 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.12281105667352676 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11078264564275742 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4466973841190338 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4595729112625122 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.34129753708839417 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9605262875556946 +jet/accuracy_3_of_5‚0.08738144487142563 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9871765971183777 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.28299781680107117 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008921721950173378 particle/accuracy_2_of_5‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 epoch‚2 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.09193257242441177 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.015508770942687988 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2871151864528656 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2341649979352951 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.15003517270088196 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34166663885116577 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7578140497207642 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3875168561935425 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3258706331253052 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3157409429550171 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3055555522441864 ("Purity/1FRt*SRqqt4FBt/event_purity‚0㣙Ž-“ ú§ @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 6 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.069444440305233 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.1988518387079239 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 8"Purity/1FRt0SRqqt*FBt/event_purity‚0.123995341360569 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10711078345775604 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.36249998211860657 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9892921447753906 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22433899343013763 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.07894736528396606 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.44771161675453186 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3033333122730255 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.3125 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8621399402618408 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9620370268821716 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00435002101585269 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.36385229229927063 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_7‚0.5 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07645847648382187 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 6 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.980008602142334 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.41939249634742737 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9873471260070801 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3054475784301758 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2866467237472534 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.04938271641731262 6 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.161896750330925 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11005154997110367 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_7‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3809758424758911 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3129967153072357 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.8755555748939514 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1291532665491104 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.0882352963089943 particle/accuracy_1_of_1‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.975712239742279 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01904761977493763 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9948257803916931 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30687594413757324 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666746139526 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3484416902065277 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03953506425023079 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16807568073272705 8"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.324735552072525 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.08755991607904434 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 jet/accuracy_2_of_7‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24703863263130188 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.34129753708839417 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.41111108660697937 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 jet/accuracy_7_of_7‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9672619104385376 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 +jet/accuracy_2_of_3‚0.18701666593551636 jet/accuracy_4_of_6‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.27869048714637756 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43358293175697327 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9970680475234985 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1096554845571518 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3364197909832001 (particle/accuracy‚0.8461232781410217 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4261505901813507 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.36666667461395264 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.028148148208856583 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 jet/accuracy_5_of_5‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3261164426803589 particle/accuracy_4_of_5‚1 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0714285746216774 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04158828780055046 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014995631761848927 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7017543911933899 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.1666666716337204 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03424396738409996 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.33725306391716003 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 8"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.499909907579422 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.4035087823867798 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4117456078529358 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.10869566351175308 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 _step‚146 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9974899291992188 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.09114398807287216 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.16572986543178558 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2666539251804352 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3898809552192688 ,jet/accuracy_3_of_4‚0.047357045114040375 particle/accuracy_2_of_7‚1 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3384050726890564 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3255474865436554 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9867740273475647 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.019095269963145256 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.47279250621795654 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.13183583319187164 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9830211997032166 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14694549143314362 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9838218092918396 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_1_of_5‚1 6validation_average_jet_accuracy‚0.3614272475242615 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300Dcoòsf403038039804e-05 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.39734262228012085 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3333333134651184 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16939488053321838 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.09824734181165695 jet/accuracy_6_of_6‚0 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34838438034057617 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.26777610182762146 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 *loss/val_total_loss‚48.059564018326675 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2806102931499481 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.013470638543367386 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.20370371639728546 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0008488964522257447 particle/accuracy_6_of_6‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15923736989498138 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6333677172660828 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.008824329823255539 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3242565393447876 particle/accuracy_3_of_4‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03536558151245117 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9931899905204773 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.33081796765327454 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9789924621582031 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2718239724636078 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9978922605514526 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 jet/accuracy_4_of_7‚0 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.37073731422424316 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.061972636729478836 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 *jet/accuracy_1_of_4‚0.8009987473487854 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9987633228302002 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3844726085662842 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.47066324949264526 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.13579638302326202 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.33888888359069824 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8063710927963257 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.43203699588775635 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 particle/accuracy_1_of_4‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_4‚0.27843645215034485 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9958208203315735 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.17123809456825256 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.18312756717205048 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3359375 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31077101826667786 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 +jet/accuracy_1_of_1‚0.39242878556251526 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9966565370559692 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16261513531208038 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9908744096755981 6 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.272583931684494 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.14294248819351196 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_2‚0.6412614583969116 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3316250741481781 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2956826090812683 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ) Purity/0FRt0SRqqt4FBt/FBt_purity‚0.75 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3584555387496948 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014283781871199608 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 trainer/global_step‚1811 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3177418112754822 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9644259810447693 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3976934552192688 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10465293377637863 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_1_of_7‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.028033796697854996 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 ,particle/sensitivity‚0.29949626326560974 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6706352233886719 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.07999999821186066 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.21785713732242584 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.34598925709724426 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0026881720405071974 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09400128573179245 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9267126321792603 jet/accuracy_5_of_7‚0 particle/accuracy_4_of_7‚1 % _timestamp‚1.7883909063952086e+09 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.15889158844947815 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.12281105667352676 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11078264564275742 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4466973841190338 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4595729112625122 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.34129753708839417 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9605262875556946 +jet/accuracy_3_of_5‚0.08738144487142563 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9871765971183777 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.28299781680107117 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008921721950173378 particle/accuracy_2_of_5‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 epoch‚2 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.09193257242441177 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.015508770942687988 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2871151864528656 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2341649979352951 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.15003517270088196 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34166663885116577 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7578140497207642 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3875168561935425 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3258706331253052 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3157409429550171 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3055555522441864 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4019857943058014 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8888888955116272 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3524305522441864 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.22304147481918335 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2848106026649475 8"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.308824747800827 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3405461013317108 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3387680947780609 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6000000238418579 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.1679399311542511 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9969950318336487 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3873119652271271 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3068452477455139 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9978922605514526 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3333333134651184 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 (particle/f_score‚0.37799370288848877 particle/accuracy_1_of_6‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.35740038752555847 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9"Purity/0FRt2SRqqt1FBt/event_purity‚0.0197132620960474 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.10879630595445633 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ) Purity/0FRt*SRqqt4FBt/FBt_purity‚0.75 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3206731379032135 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.16128724813461304 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24217171967029572 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.038148149847984314 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9473055601119995 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 6 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.800000011920929 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3142682909965515 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.981205403804779 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8818008899688721 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.09114398807287216 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 particle/accuracy_3_of_3‚1 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1257437914609909 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.28553009033203125 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.13106638193130493 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3354838490486145 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2719298303127289 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1944444179534912 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.005063365679234266 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9867998361587524 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27742448449134827 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17851795256137848 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.21581311523914337 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2716553509235382 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9858120083808899 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2514374256134033 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.30906835198402405 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2914920747280121 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3333333134651184 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9933744668960571 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.30399808287620544 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.06871610134840012 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33166345953941345 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15756537020206451 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.8855159878730774 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.42927730083465576 6 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.147939071059227 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9837575554847717 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.31585487723350525 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9855555295944214 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16803213953971863 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11499813199043274 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9765745401382446 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.998068630695343 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.29101210832595825 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.04597701132297516 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.048966627568006516 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08291006088256836 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9736841917037964 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 _runtime‚1662.152945186 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.7916666865348816 particle/accuracy_2_of_6‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08973724395036697 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_5‚1 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.30577200651168823 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2887383699417114 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5333333611488342 ,jet/accuracy_4_of_5‚0.023156682029366493 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3749999701976776 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3628650903701782 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 particle/accuracy_4_of_6‚1 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3946434259414673 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11730237305164337 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.11806695908308029 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_3‚0.01566033437848091 particle/accuracy_5_of_5‚1 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1525263488292694 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.719298243522644 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.32894736528396606 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.25132274627685547 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.19469699263572693’jKrö›” ‚(: ýÛâÔš’¸u gpu.0.gpu‚4.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚29.0 gpu.0.powerWatts‚75.668' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚15.133600000000003 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0eÿÚ-• ‚(:¢ ýÛâÔøÝÙw disk.drbd1777.in‚0% disk./.usageGB‚1044.2495002746582 disk.rbd3.out‚ 1.36328125" disk.dm-0.out‚9473.91357421875 disk.sda3.out‚ 68.6640625! disk.rbd2.out‚135.25048828125 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.dm-1.in‚0.109375' disk.nvme0c0n1.out‚9473.91357421875 disk.sda2.out‚0 disk.sda3.in‚0.109375 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875 disk.rbd2.in‚0$ memory_percent‚87.80314922332764 disk.sda1.in‚0 disk.nvme0c0n1.in‚64.5! proc.memory.rssMB‚ 2910.265625 proc.cpu.threads‚17 network.sent‚ 3.408082e+06 disk.rbd3.in‚0 disk.dm-0.in‚64.5 disk.dm-1.out‚ 68.6640625% disk.nvme0n1.out‚9473.91357421875 disk.sda2.in‚0) proc.memory.availableMB‚ 1998.33203125) proc.memory.percent‚17.76285171508789 disk.sda.in‚0.109375 disk.sda1.out‚0 cpu‚24.94799086224164 network.recv‚603372( disk./.usagePercent‚70.0804601471228 disk.nvme0n1.in‚64.5 disk.sda.out‚ 68.6640625 disk.rbd0.in‚0¯'RÁ‘– ‚(:† ŒÜâÔ§ŠÊu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.509' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.9018 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶Ex`1— ‚(:¦ ŒÜâÔ®‡’x disk.sda2.in‚0 disk.nvme0n1.in‚64.5 proc.cpu.threads‚17 disk.rbd3.in‚0 disk.rbd1.in‚0$ memory_percent‚87.79504299163818) proc.memory.availableMB‚ 1999.66015625 disk.sda1.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-0.in‚64.5 disk.rbd2.in‚0% disk.nvme0n1.out‚9572.80224609375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125! proc.memory.rssMB‚ 2909.265625 disk.sda1.out‚0' disk.nvme0c0n1.out‚9572.80224609375 disk.sda3.in‚0.109375 disk.rbd1.out‚ 0.982421875 disk.sda2.out‚0" disk.dm-0.out‚9572.80224609375 disk.dm-1.in‚0.109375 disk.sda.out‚ 68.91015625% disk./.usageGB‚1044.2495193481445 disk.drbd1777.out‚0 cpu‚24.953879329152066 network.sent‚ 3.51157e+06 network.recv‚610040 disk.drbd1777.in‚0 disk.nvme0c0n1.in‚64.5 disk.sda3.out‚ 68.91015625 disk.dm-1.out‚ 68.91015625! disk.rbd2.out‚136.40283203125 disk.sda.in‚0.109375) disk./.usagePercent‚70.08046142716049) proc.memory.percent‚17.75674819946289U/¸Ë’˜ ‚(:‡ ›ÜâÔîÛ£u gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚154.969' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.9938 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ùø©(™ ‚(: ›ÜâÔ‘Îúx network.recv‚615125 disk.sda2.in‚0 network.sent‚ 3.518141e+06! disk.rbd2.out‚136.43408203125 disk.rbd3.in‚0" disk.dm-0.out‚9814.02490234375$ memory_percent‚88.67132663726807) disk./.usagePercent‚70.08041764987215% disk.nvme0n1.out‚9814.02490234375 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚9814.02490234375 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.36328125 disk.dm-1.out‚69.59375) proc.memory.availableMB‚ 1856.08984375! proc.memory.rssMB‚ 2909.265625 disk.sda1.out‚0 disk.rbd0.in‚0 disk.sda3.in‚0.109375 disk.rbd1.in‚0) proc.memory.percent‚17.75674819946289 disk.dm-0.in‚64.5 disk.sda3.out‚69.59375 disk.dm-1.in‚0.109375 disk.nvme0n1.in‚64.5 disk.sda.out‚69.59375 disk.rbd1.out‚ 0.982421875$ disk./.usageGB‚1044.248867034912 disk.sda1.in‚0 disk.nvme0c0n1.in‚64.5 cpu‚24.959797832815145 proc.cpu.threads‚17/ I&2š - _wandbruntime‚1695  _runtime‚1695† ­Ç› Á & lr-AdamW/pg1‚0.0015055940704549178 & lr-AdamW/pg2‚0.0015055940704549178 trainer/global_step‚1849 % _timestamp‚1.7883909401439252e+09 _runtime‚1695.899903795 _step‚147Es8ºÌœ Æ & lr-AdamW/pg1‚0.0015055940704549178 & lr-AdamW/pg2‚0.0015055940704549178 trainer/global_step‚1849 % _timestamp‚1.7883909401439252e+09 _runtime‚1695.899903795 _step‚147“ÔÊ#<2 - _wandbruntime‚1695  _runtime‚1695µû[Šþž ÛO P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚124642931627008 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚123127173668162 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚71224061 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1598323047936 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1526177474048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚126169305280000 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1526373652992 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 152617†(Cã!7474048 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31755970 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9927134 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31755972 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19464456 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9927643 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256543744 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 1695.90119623 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚124642914587648 % _timestamp‚1.7883909401449525e+09 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚124642914587648 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39468091 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19464459 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1517862982978 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126169092061696 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚127245816373248 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1517670986438 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚126169305280000 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126169092061696 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1526373652992 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚29391590 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚127245559829504 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚125647261685248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚121609293645824 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚125647236781568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231640064 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚39490689 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚39490689 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚123126964632262 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 _step‚148 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31755970 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0.m4Mæ'Ÿ àO M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚71246661 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31755972 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚39468091 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚509 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126169092061696 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚127245816373248 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1517670986438 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚126169305280000 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126169092061696 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1526373652992 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚29391590 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚127245559829504 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚125647261685248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚121609293645824 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚125647236781568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231640064 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚39490689 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚39490689 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚123126964632262 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 _step‚148 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31755970 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚124642931627008 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚123127173668162 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚71224061 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1598323047936 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1526177474048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚126169305280000 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1526373652992 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1526177474048 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31755970 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9927134 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31755972 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19464456 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9927643 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256543744 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 1695.90119623 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚124642914587648 % _timestamp‚1.7883909401449525e+09 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚124642914587648 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39468091 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19464459 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1517862982978 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1526975864832 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29406852 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39511587 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71284079 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1527172057088 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31772492 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19474490 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚126234929489408 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚149 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1518656866134 _runtime‚1696.651867341 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71261453 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9932905 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123191566935894 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748òÉö¢ ëM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71261453 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9932905 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123191566935894 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚71261453 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚126235142721024 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1526975864832 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29406852 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39511587 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71284079 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1527172057088 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31772492 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19474490 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚126234929489408 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚149 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1518656866134 _runtime‚1696.651867341•”7çL2£ - _wandbruntime‚1696  _runtime‚1696“àBW¤ Ñ 0loss/FBt2/assignment_loss‚0.2165255523814886 /loss/FRt4/detection_loss‚2.5533106699585915 /loss/FBt2/detection_loss‚0.6075518959926234 %loss/total_loss‚47.86446934883314 0loss/FRt1/assignment_loss‚3.3871379969995234 0loss/FRt3/assignment_loss‚3.2470029021564284 epoch‚3 % _timestamp‚1.7883909408972194e+09 _step‚150 trainer/global_step‚1849 .loss/FRt2/detection_loss‚1.627444223344031 /loss/FRt2/assignment_loss‚2.032213590522804 1loss/FBt1/assignment_loss‚0.16739889088785276 1loss/FBt4/assignment_loss‚0.19890024830215225 /loss/FRt3/detection_loss‚2.3383412207427776 1loss/SRqqt2/detection_loss‚1.0575319329133401 1loss/SRqqt3/assignment_loss‚1.666302513532542 1loss/SRqqt4/detection_loss‚2.4319708549207255 .loss/FBt4/detection_loss‚3.346961561371298 _runtime‚1696.653009633 2loss/SRqqt2/assignment_loss‚1.7286053170951514 0loss/FBt3/assignment_loss‚0.5605012684574594 /loss/FRt1/detection_loss‚2.6513632483120206 1loss/SRqqt3/detection_loss‚1.8377270287758596 .loss/FBt3/detection_loss‚2.445815586525461 /loss/FRt4/assignment_loss‚4.461564090516832 1loss/SRqqt1/assignment_loss‚1.942578118867599 1loss/SRqqt4/assignment_loss‚2.531494266563846 0loss/SRqqt1/detection_loss‚1.823682439728425 .loss/FBt1/detection_loss‚3.002543929964304pÃMe\¥ Ö /loss/FRt2/assignment_loss‚2.032213590522804 1loss/FBt1/assignment_loss‚0.16739889088785276 1loss/FBt4/assignment_loss‚0.19890024830215225 /loss/FRt3/detection_loss‚2.3383412207427776 1loss/SRqqt2/detection_loss‚1.0575319329133401 1loss/SRqqt3/assignment_loss‚1.666302513532542 1loss/SRqqt4/detection_loss‚2.4319708549207255 .loss/FBt4/detection_loss‚3.346961561371298 _runtime‚1696.653009633 2loss/SRqqt2/assignment_loss‚1.7286053170951514 0loss/FBt3/assignment_loss‚0.5605012684574594 /loss/FRt1/detection_loss‚2.6513632483120206 1loss/SRqqt3/detection_loss‚1.8377270287758596 .loss/FBt3/detection_loss‚2.445815586525461 /loss/FRt4/assignment_loss‚4.461564090516832 1loss/SRqqt1/assignment_loss‚1.942578118867599 1loss/SRqqt4/assignment_loss‚2.531494266563846 0loss/SRqqt1/detection_loss‚1.823682439728425 .loss/FBt1/detection_loss‚3.002543929964304 0loss/FBt2/assignment_loss‚0.2165255523814886 /loss/FRt4/detection_loss‚2.5533106699585915 /loss/FBt2/detection_loss‚0.6075518959926234 %loss/total_loss‚47.86446934883314 0loss/FRt1/assignment_loss‚3.3871379969995234 0loss/FRt3/assignment_loss‚3.2470029021564284 epoch‚3 % _timestamp‚1.7883909408972194e+09 _step‚150 trainer/global_step‚1849 .loss/FRt2/detection_loss‚1.627444223344031–$t¤Ó›¦ ‚(: ªÜâÔ®à¸v gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.8' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.560000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0†KÔq-§ ‚(:¢ ªÜâÔ¬‡¥y disk.dm-0.in‚64.5) proc.memory.percent‚17.75674819946289 disk.sda3.out‚ 70.15234375 disk.rbd1.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 network.recv‚619012 disk.rbd2.in‚0 disk.rbd0.out‚0.078125$ memory_percent‚88.67321014404297! proc.memory.rssMB‚ 2909.265625 cpu‚24.965494572125184 network.sent‚ 3.523402e+06 disk.sda.in‚0.109375 disk.nvme0c0n1.in‚64.5 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.sda.out‚ 70.15234375 disk.rbd3.out‚ 1.36328125 proc.cpu.threads‚17) disk./.usagePercent‚70.08045707503241 disk.rbd1.out‚ 0.982421875 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.109375$ disk./.usageGB‚1044.249454498291 disk.rbd2.out‚137.5517578125 disk.drbd1777.out‚0 disk.dm-1.out‚ 70.15234375' disk.nvme0c0n1.out‚9815.53076171875" disk.dm-0.out‚9815.53076171875& proc.memory.availableMB‚ 1855.78125 disk.nvme0n1.in‚64.5% disk.nvme0n1.out‚9815.53076171875Dä¹>¨ ‚(:’ ¹ÜâÔšŒ»u gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.672' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.534399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÑNë*© ‚(:Ÿ ¹ÜâÔ“Ѩx disk.sda2.out‚0# proc.memory.rssMB‚ 2909.26953125 network.recv‚624140 disk.rbd1.out‚ 0.982421875 disk.drbd1777.in‚0# memory_percent‚88.6728048324585 disk.rbd2.out‚138.5947265625 disk.nvme0c0n1.in‚64.5 proc.cpu.threads‚17 disk.sda3.out‚ 70.984375 disk.sda.out‚ 70.984375( proc.memory.percent‚17.7567720413208) disk./.usagePercent‚70.08045579499473% disk./.usageGB‚1044.2494354248047" disk.dm-0.out‚9913.98486328125 disk.rbd2.in‚0 disk.sda1.out‚0' disk.nvme0c0n1.out‚9913.98486328125 cpu‚24.97127136536917 disk.dm-1.out‚ 70.984375 network.sent‚ 3.610219e+06 disk.dm-0.in‚64.5 disk.sda3.in‚0.109375 disk.rbd1.in‚0 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-1.in‚0.109375 disk.sda1.in‚0 disk.rbd0.in‚0) proc.memory.availableMB‚ 1855.84765625 disk.nvme0n1.in‚64.5% disk.nvme0n1.out‚9913.98486328125 disk.drbd1777.out‚0? ]2ª - _wandbruntime‚1737  _runtime‚1737Ž±2Å« ¿ trainer/global_step‚1899 % _timestamp‚1.7883909818781502e+09 _runtime‚1737.634140701 _step‚151 % lr-AdamW/pg1‚0.001488962836167629 % lr-AdamW/pg2‚0.001488962836167629=+ʬ Ä trainer/global_step‚1899 % _timestamp‚1.7883909818781502e+09 _runtime‚1737.634140701 _step‚151 % lr-AdamW/pg1‚0.001488962836167629 % lr-AdamW/pg2‚0.001488962836167629—ÄQaG2­ - _wandbruntime‚1737  _runtime‚1737ܯUâä'® ÞO RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40515043 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚129461204493824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚130564434798080 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40537641 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129460991275520 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1557584302386 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚124789262874624 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 % _timestamp‚1.7883909818791823e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚127894866434048 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126346864216370 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126346655180470 _step‚152 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚127894866434048 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚128923511597568 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1640898296832 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚127894883473408 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚73119613 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40537641 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1566124841472 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚30151399 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129461204493824 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚127894883473408 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19966156 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚1737.638925476 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚73097013 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32581970 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32581972 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507ÌBEX'¯ ãO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40537641 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129460991275520 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1557584302386 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚124789262874624 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 % _timestamp‚1.7883909818791823e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚127894866434048 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126346864216370 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126346655180470 _step‚152 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚127894866434048 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚128923511597568 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1640898296832 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚127894883473408 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚73119613 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40537641 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1566124841472 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚30151399 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129461204493824 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚127894883473408 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19966156 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚1737.638925476 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚73097013 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32581970 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32581972 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40515043 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚129461204493824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚130564434798080 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.oxÀÎô‘n_train_batch_start/active_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0˜‹²M2° - _wandbruntime‚1738  _runtime‚1738ƒfí&± çM cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚128989061997568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚127959905470976 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚127959905470976 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10190551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚124852879298560 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚129526828703232 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30166741 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚126411257484102 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1558378185542 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚153 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1558186188794 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚73157031 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚566 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚130630551249408 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1566923232256 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1641749976576 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 trainer/global_step‚1899 _runtime‚1738.385362151 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1567119424512 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1566923232256 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40535915 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚128989037093888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30167310 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10191117 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 |R×ò&² ìM EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 trainer/global_step‚1899 _runtime‚1738.385362151 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1567119424512 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1566923232256 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40535915 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚128989037093888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30167310 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10191117 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚128989061997568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚127959905470976 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚127959905470976 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10190551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚124852879298560 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚129526828703232 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30166741 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚126411257484102 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1558378185542 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚153 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1558186188794 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚73157031 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚566 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚130630551249408 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1566923232256 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1641749976576 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 Bs1· ‚(:¦ ÈÜâÔ—Æ°x disk.rbd2.in‚0) disk./.usagePercent‚70.08082956599473 disk.sda1.out‚0 disk.rbd3.in‚0$ memory_percent‚88.67268562316895( proc.memory.availableMB‚ 1855.8671875# proc.memory.rssMB‚ 2909.26953125 disk.rbd1.in‚0 network.sent‚ 3.616414e+06 disk.sda.in‚0.109375 disk.nvme0c0n1.in‚64.5' disk.nvme0c0n1.out‚10019.2041015625 disk.sda2.out‚0( proc.memory.percent‚17.7567720413208 cpu‚24.976789409647694 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.sda3.in‚0.109375 disk.dm-1.out‚ 71.39453125 disk.sda.out‚ 71.39453125 disk.dm-0.in‚64.5" disk.dm-0.out‚10019.2041015625% disk.nvme0n1.out‚10019.2041015625 proc.cpu.threads‚17 disk.rbd1.out‚ 0.982421875% disk./.usageGB‚1044.2550048828125 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.rbd2.out‚139.7509765625 disk.nvme0n1.in‚64.5 disk.rbd0.in‚0 disk.rbd3.out‚ 1.36328125 disk.dm-1.in‚0.109375 network.recv‚628733 disk.sda2.in‚0 disk.sda3.out‚ 71.39453125í¦¸ ‚(:’ ×ÜâÔ—êÑu gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.319' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.863799999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0”¦É71¹ ‚(:¦ ×ÜâÔÐÄÝw disk.dm-0.in‚64.5 disk.sda2.out‚0$ memory_percent‚88.67475986480713# proc.memory.rssMB‚ 2909.26953125( proc.memory.percent‚17.7567720413208 cpu‚24.982372849951034 network.recv‚635175 disk.dm-1.in‚0.109375 disk.drbd1777.out‚0 proc.cpu.threads‚17% disk./.usageGB‚1044.2652740478516 disk.rbd3.in‚0 disk.rbd1.in‚0' disk.nvme0c0n1.out‚10030.4970703125 disk.sda2.in‚0( disk./.usagePercent‚70.0815187382769 disk.rbd3.out‚ 1.36328125 disk.sda1.out‚0 network.sent‚ 3.704304e+06% disk.nvme0n1.out‚10030.4970703125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.dm-1.out‚ 72.11328125 disk.nvme0n1.in‚64.5 disk.rbd0.out‚0.078125 disk.sda3.in‚0.109375 disk.sda3.out‚ 72.11328125 disk.rbd1.out‚ 0.982421875 disk.rbd2.in‚0 disk.rbd2.out‚139.7822265625 disk.sda.out‚ 72.11328125 disk.nvme0c0n1.in‚64.5) proc.memory.availableMB‚ 1855.52734375" disk.dm-0.out‚10030.4970703125 disk.sda.in‚0.109375ÆJ®¹‘º ‚(:† æÜâÔ´Œèu gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚91.401' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.2802 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Bh—q2» ‚(:§ æÜâÔö³Åx" disk.dm-0.out‚10130.9501953125 disk.sda3.out‚ 72.42578125$ memory_percent‚88.67475986480713 cpu‚24.987693343050115' disk.nvme0c0n1.out‚10130.9501953125 disk.nvme0n1.in‚64.5( proc.memory.percent‚17.7567720413208 proc.cpu.threads‚17 network.recv‚639014 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.out‚ 72.42578125 disk.rbd2.in‚0) proc.memory.availableMB‚ 1855.52734375 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.dm-1.in‚0.109375 disk.rbd2.out‚140.8720703125% disk.nvme0n1.out‚10130.9501953125 network.sent‚ 3.709612e+06 disk.rbd0.out‚0.078125 disk.rbd3.out‚ 1.36328125% disk./.usageGB‚1044.2649993896484 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚64.5 disk.sda.in‚0.109375 disk.sda1.in‚0# proc.memory.rssMB‚ 2909.26953125 disk.rbd1.out‚ 0.982421875 disk.sda.out‚ 72.42578125) disk./.usagePercent‚70.08150030573444 disk.nvme0c0n1.in‚64.5 disk.sda3.in‚0.109375 disk.rbd1.in‚0 disk.sda1.out‚0uSØ2¼ - _wandbruntime‚1779  _runtime‚1779ÝîtIĽ ¾ _runtime‚1779.684012464 _step‚155 % lr-AdamW/pg1‚0.001471805873272377 % lr-AdamW/pg2‚0.001471805873272377 trainer/global_step‚1949 $ _timestamp‚1.788391023928018e+09' èþɾ à _step‚155 % lr-AdamW/pg1‚0.001471805873272377 % lr-AdamW/pg2‚0.001471805873272377 trainer/global_step‚1949 $ _timestamp‚1.788391023928018e+09 _runtime‚1779.684012464›rÊ’2¿ - _wandbruntime‚1779  _runtime‚1779s:X:²À ÝO YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚132753103707648 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41584593 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚132199811317248 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚129566345728678 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33407970 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _step‚156 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20467856 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1606072208896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10445554 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚131146818280448 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚131146818280448 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1597305621794 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚132752890489344 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1682940221440 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚132753103707648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G ι1?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _runtime‚1779.689100491 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33407972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20467859 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚132752890489344 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30913413œúÉhè'Á âO ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1606268387840 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1597113625254 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _runtime‚1779.689100491 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33407972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20467859 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚132752890489344 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30913413 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚132753103707648 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41584593 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚132199811317248 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚129566345728678 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33407970 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 _step‚156 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20467856 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1606072208896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10445554 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚131146818280448 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚131146818280448 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1597305621794 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚132752890489344 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1682940221440 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚132753103707648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 IDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75029983 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚133949077282304 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚132818941148672 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚546 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚157 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30928185 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1607066791936 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚128032831488000 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚1949 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚131211874356736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33424492 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30928731 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1597907508202 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41582867 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚133949350589952 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20477890 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33424492 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚131211857317376 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1607066791936 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚128032848527360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚75029983 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75007357 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 $ _timestamp‚1.788391024678168e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1606870599680 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10450295 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1598099504950 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824vfpl¥Ä ìM NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41582867 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41605491 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚132818727917056 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75029983 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚133949077282304 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚132818941148672 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚546 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚157 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30928185 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1607066791936 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚128032831488000 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚1949 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚131211874356736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33424492 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30928731 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1597907508202 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41582867 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚133949350589952 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20477890 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33424492 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚131211857317376 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1607066791936 FDeviceStatsMonitor.on_train_batch_e«Û‰¿M nd/active_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚128032848527360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚75029983 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75007357 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 $ _timestamp‚1.788391024678168e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1606870599680 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10450295 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1598099504950 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33424490 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1684013776384 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚132818941148672 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1606870599680 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚132265336813568 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚129630948032310 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚131211857317376 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10450838 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚1296307389962028˜ÜÁ2Å - _wandbruntime‚1780  _runtime‚1780Èñ Z\Æ Ö /loss/FRt1/assignment_loss‚3.789293583076779 /loss/FRt4/assignment_loss‚4.019391708903843 1loss/SRqqt3/detection_loss‚1.9613327926823072 %loss/total_loss‚46.20270911793499 % _timestamp‚1.7883910246801853e+09 _step‚158 2loss/SRqqt3/assignment_loss‚1.8654727890555347 /loss/FRt2/detection_loss‚1.7569908401837535 2loss/SRqqt2/assignment_loss‚1.3848804153389949 /loss/FBt1/detection_loss‚2.2876875791698694 /loss/FBt2/detection_loss‚0.5980596047311281 /loss/FRt3/assignment_loss‚3.653282745016946 1loss/FBt1/assignment_loss‚0.38169732004826074 1loss/FBt3/assignment_loss‚0.22541395660333696 1loss/SRqqt2/detection_loss‚1.0202248638733886 1loss/SRqqt4/detection_loss‚2.2883529985273205 /loss/FBt3/detection_loss‚2.5550316057421942 trainer/global_step‚1949 1loss/SRqqt1/assignment_loss‚1.586645454996162 .loss/FRt1/detection_loss‚2.364689231804102 0loss/FBt4/assignment_loss‚0.5591943508130498 /loss/FBt4/detection_loss‚2.6771176785230635 0loss/FRt2/assignment_loss‚1.9319521679597742 1loss/FBt2/assignment_loss‚0.20865904921072553 /loss/FRt3/detection_loss‚2.6983271261884108 0loss/SRqqt1/detection_loss‚1.852267248461945 _runtime‚1780.435993556 2loss/SRqqt4/assignment_loss‚2.0913016400929236 /loss/FRt4/detection_loss‚2.4454423669311733 epoch‚3ˆ½Í©aÇ Û _runtime‚1780.435993556 2loss/SRqqt4/assignment_loss‚2.0913016400929236 /loss/FRt4/detection_loss‚2.4454423669311733 epoch‚3 /loss/FRt1/assignment_loss‚3.789293583076779 /loss/FRt4/assignment_loss‚4.019391708903843 1loss/SRqqt3/detection_loss‚1.9613327926823072 %loss/total_loss‚46.20270911793499 % _timestamp‚1.7883910246801853e+09 _step‚158 2loss/SRqqt3/assignment_loss‚1.8654727890555347 /loss/FRt2/detection_loss‚1.7569908401837535 2loss/SRqqt2/assignment_loss‚1.3848804153389949 /loss/FBt1/detection_loss‚2.2876875791698694 /loss/FBt2/detection_loss‚0.5980596047311281 /loss/FRt3/assignment_loss‚3.653282745016946 1loss/FBt1/assignment_loss‚0.38169732004826074 1loss/FBt3/assignment_loss‚0.22541395660333696 1loss/SRqqt2/detection_loss‚1.0202248638733886 1loss/SRqqt4/detection_loss‚2.2883529985273205 /loss/FBt3/detection_loss‚2.5550316057421942 trainer/global_step‚1949 1loss/SRqqt1/assignment_loss‚1.586645454996162 .loss/FRt1/detection_loss‚2.364689231804102 0loss/FBt4/assignment_loss‚0.5591943508130498 /loss/FBt4/detection_loss‚2.6771176785230635 0loss/FRt2/assignment_loss‚1.9319521679597742 1loss/FBt2/assignment_loss‚0.20865904921072553 /loss/FRt3/detection_loss‚2.6983271261884108 0loss/SRqqt1/detection_loss‚1.852267248461945žöT´4È ‚(:’ õÜâÔ¯Õ½v gpu.0.gpu‚28.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚153.616' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.723200000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‘l#¡É ‚(: õÜâÔöè½x disk.nvme0c0n1.in‚64.5 disk.rbd3.out‚ 1.36328125! disk.dm-0.out‚10132.427734375 cpu‚24.992953056992157 disk.rbd3.in‚0 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125& disk.nvme0c0n1.out‚10132.427734375) proc.memory.availableMB‚ 1855.26953125# proc.memory.rssMB‚ 2909.26953125( proc.memory.percent‚17.7567720413208 network.sent‚ 3.715053e+06 disk.sda3.out‚73.25 disk.sda2.in‚0 disk.sda2.out‚0# memory_percent‚88.6763334274292 proc.cpu.threads‚17 network.recv‚642511 disk.rbd1.out‚ 0.982421875 disk.drbd1777.out‚0$ disk./.usageGB‚1044.266689300537 disk.rbd2.in‚0 disk.nvme0n1.in‚64.5 disk.dm-1.out‚73.25 disk.rbd2.out‚141.9775390625 disk.sda.in‚0.109375 disk.sda.out‚73.25) disk./.usagePercent‚70.08161371707212 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0$ disk.nvme0n1.out‚10132.427734375 disk.dm-0.in‚64.5CMG’Ê ‚(:‡ „ÝâÔ¸º‡v gpu.0.gpu‚41.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.201' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.4402 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*˜¼7,Ë ‚(:¡ „ÝâÔ…£æy proc.cpu.threads‚17 disk.dm-0.in‚64.5 disk.sda.out‚ 73.5859375 disk.sda1.in‚0' disk.nvme0c0n1.out‚10231.2822265625 disk.sda2.out‚0! disk.rbd2.out‚142.13037109375 disk.rbd0.in‚0 disk.dm-1.out‚ 73.5859375 disk.rbd3.in‚0" disk.dm-0.out‚10231.2822265625 disk.sda3.out‚ 73.5859375 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.nvme0n1.in‚64.5% disk.nvme0n1.out‚10231.2822265625 disk.sda1.out‚0( proc.memory.availableMB‚ 1855.3828125 network.sent‚ 3.80265e+06% disk./.usageGB‚1044.2665939331055 disk.rbd3.out‚ 1.36328125 disk.rbd2.in‚0 disk.nvme0c0n1.in‚64.5# memory_percent‚88.6756420135498 cpu‚24.99822394708414) disk./.usagePercent‚70.08160731688375 disk.sda3.in‚0.109375 disk.rbd1.in‚0# proc.memory.rssMB‚ 2909.26953125 network.recv‚648875 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125 disk.dm-1.in‚0.109375 disk.rbd1.out‚ 0.982421875 disk.drbd1777.in‚0( proc.memory.percent‚17.7567720413208¿…#Ì ‚(:’ “ÝâÔ˜Ó»u gpu.0.gpu‚38.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.496' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.099200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0p¨.Í ‚(:£ “ÝâÔÌæ±y disk.drbd1777.in‚0 disk.rbd1.out‚ 0.982421875 disk.sda1.out‚0 cpu‚25.003458849497548 proc.cpu.threads‚17) disk./.usagePercent‚70.08164034185566% disk./.usageGB‚1044.2670860290527% disk.nvme0n1.out‚10329.8642578125 disk.sda.in‚0.109375$ memory_percent‚88.67416381835938' disk.nvme0c0n1.out‚10329.8642578125 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚ 74.08984375 disk.rbd1.in‚0! disk.rbd2.out‚143.23583984375 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.dm-1.out‚ 74.08984375 disk.rbd0.out‚0.078125# proc.memory.rssMB‚ 2909.26953125 network.recv‚653423 disk.rbd0.in‚0 disk.nvme0c0n1.in‚64.5 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚64.5 disk.sda.out‚ 74.08984375 disk.sda2.out‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚64.5$ proc.memory.availableMB‚1855.625 network.sent‚ 3.809062e+06" disk.dm-0.out‚10329.8642578125 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375( proc.memory.percent‚17.7567720413208~6ÝÔ2Î - _wandbruntime‚1821  _runtime‚1821¸öýÇÏ Á & lr-AdamW/pg1‚0.0014541375066510743 & lr-AdamW/pg2‚0.0014541375066510743 trainer/global_step‚1999 % _timestamp‚1.7883910657995286e+09 _runtime‚1821.555628386 _step‚159+äX{ÌÐ Æ & lr-AdamW/pg2‚0.0014541375066510743 trainer/global_step‚1999 % _timestamp‚1.7883910657995286e+09 _runtime‚1821.555628386 _step‚159 & lr-AdamW/pg1‚0.0014541375066510743ŸæFï÷2Ñ - _wandbruntime‚1821  _runtime‚1821Ò)+¯ã'Ò ÝO G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚76842917 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚134398770126848 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚132786245312786 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚136045002921472 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚134398787166208 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42608947 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚1999 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚137201397683712 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚131149201332224 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1646215755264 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚34233972 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚545 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34233972 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1636834944662 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚1821.559064801 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42631545 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42608947 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚136045002921472 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚136044789703168 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1725311550464¿#ê;è'Ó âO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1646215755264 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1646019576320 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚131149218371584 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚134398770126848 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10704519 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1637026941202 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚134398787166208 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20969559 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚34233972 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚545 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34233972 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1636834944662 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚1821.559064801 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42631545 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42608947 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚136045002921472 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚136044789703168 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1725311550464 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚76842917 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚134398770126848 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚132786245312786 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚136045002921472 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚134398787166208 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42608947 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚1999 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚137201397683712 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚131149201332224 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1646215755264 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _runtime‚1822.539470558 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚136110840362496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1647014159360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34250490 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚134463826203136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271210496 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚161 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚1g¢;ŸÝ 36110840362496 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚134463809163776 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76902935 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31689218 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚76880309 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚136110627130880 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1726158165504 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42652443 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1637628827610 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10710163 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20979590 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1725911858688 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚132850638580518 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚131212817756160 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚42629819 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚131212800716800 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1637820824358 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20979593 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚137267769795072 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚132850429544410 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚1999 % _timestamp‚1.7883910667828946e+09 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚134463809163776 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚76880309 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34250492 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42652443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚135541586725888 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚135541611629568 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76902935 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚134463826203136 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚137267498584576 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚76880309 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚136110627130880 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1726158165504 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42652443 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1637628827610 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10710163 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20979590 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1725911858688 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚132850638580518 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚131212817756160 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚42629819 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚131212800716800 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1637820824358 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20979593 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚137267769795072 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚132850429544410 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚1999 % _timestamp‚1.7883910667828946e+09 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚134463809163776 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚76880309 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34250492 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42652443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚135541586725888 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚135541611629568 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76902935 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚134463826203136 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚137267498584576 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _runtime‚1822.539470558 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚136110840362496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1647014159360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34250490 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚134463826203136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271210496 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚161 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚136110840362496 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚134463809163776 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76902935 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31689218 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 F[]Ø × _runtime‚ 1822.54057575 1loss/FBt1/assignment_loss‚0.48038923938655387 0loss/FBt4/assignment_loss‚0.4664166499860585 0loss/FRt1/assignment_loss‚3.5851335261997423 2loss/SRqqt4/assignment_loss‚2.0822760808089424 /loss/FRt1/detection_loss‚2.4662979565168683 /loss/FRt4/assignment_loss‚4.322451691417133 1loss/SRqqt3/detection_loss‚2.2262879949063064 1loss/SRqqt4/detection_loss‚2.3068406402118624 .loss/FBt3/detection_loss‚3.006906885367173 .loss/FBt4/detection_loss‚4.912648071845372 0loss/FRt3/assignment_loss‚3.3719659011031307 2loss/SRqqt3/assignment_loss‚1.7744152600876988 1loss/FBt3/assignment_loss‚0.29578478765100813 /loss/FRt2/detection_loss‚1.9247105126579602 epoch‚3 % _timestamp‚1.7883910667847898e+09 0loss/FRt2/assignment_loss‚2.4752034359508093 _step‚162 1loss/FBt2/assignment_loss‚0.13677415553400601 /loss/FRt3/detection_loss‚2.5163146104923513 .loss/FRt4/detection_loss‚2.507896629326484 1loss/SRqqt1/detection_loss‚1.8234462442723187 1loss/SRqqt2/detection_loss‚0.9104710727818867 /loss/FBt1/detection_loss‚2.0968723334372044 /loss/FBt2/detection_loss‚0.7811407453027265 %loss/total_loss‚49.67927654252118 2loss/SRqqt1/assignment_loss‚1.7840415370565923 2loss/SRqqt2/assignment_loss‚1.4245905802209875 trainer/global_step‚1999MÒIrbÙ Ü % _timestamp‚1.7883910667847898e+09 0loss/FRt2/assignment_loss‚2.4752034359508093 _step‚162 1loss/FBt2/assignment_loss‚0.13677415553400601 /loss/FRt3/detection_loss‚2.5163146104923513 .loss/FRt4/detection_loss‚2.507896629326484 1loss/SRqqt1/detection_loss‚1.8234462442723187 1loss/SRqqt2/detection_loss‚0.9104710727818867 /loss/FBt1/detection_loss‚2.0968723334372044 /loss/FBt2/detection_loss‚0.7811407453027265 %loss/total_loss‚49.67927654252118 2loss/SRqqt1/assignment_loss‚1.7840415370565923 2loss/SRqqt2/assignment_loss‚1.4245905802209875 trainer/global_step‚1999 _runtime‚ 1822.54057575 1loss/FBt1/assignment_loss‚0.48038923938655387 0loss/FBt4/assignment_loss‚0.4664166499860585 0loss/FRt1/assignment_loss‚3.5851335261997423 2loss/SRqqt4/assignment_loss‚2.0822760808089424 /loss/FRt1/detection_loss‚2.4662979565168683 /loss/FRt4/assignment_loss‚4.322451691417133 1loss/SRqqt3/detection_loss‚2.2262879949063064 1loss/SRqqt4/detection_loss‚2.3068406402118624 .loss/FBt3/detection_loss‚3.006906885367173 .loss/FBt4/detection_loss‚4.912648071845372 0loss/FRt3/assignment_loss‚3.3719659011031307 2loss/SRqqt3/assignment_loss‚1.7744152600876988 1loss/FBt3/assignment_loss‚0.29578478765100813 /loss/FRt2/detection_loss‚1.9247105126579602 epoch‚3¢ ðkÚ ‚(:… ¢ÝâÔê²Óu gpu.0.gpu‚48.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚95.485' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.097 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.Ýç.Û ‚(:£ ¢ÝâÔµ»Íx# disk.nvme0c0n1.out‚ 10339.609375 disk.sda2.in‚0 disk.rbd2.in‚0 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.dm-0.out‚ 10339.609375) proc.memory.availableMB‚ 1855.58203125) disk./.usagePercent‚70.08221610280017 disk.rbd3.out‚ 1.36328125 disk.sda3.out‚74.6875 disk.rbd1.out‚ 0.982421875 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.rbd1.in‚0# proc.memory.rssMB‚ 2909.26953125 network.sent‚ 3.814318e+06! disk.nvme0n1.out‚ 10339.609375 disk.drbd1777.out‚0% disk./.usageGB‚1044.2756652832031 disk.sda.out‚74.6875 disk.dm-0.in‚ 71.57421875$ memory_percent‚88.67442607879639( proc.memory.percent‚17.7567720413208 cpu‚25.008234603011037 network.recv‚657312 disk.nvme0n1.in‚ 71.57421875 disk.sda1.in‚0 disk.dm-1.out‚74.6875 disk.rbd2.out‚ 144.396484375 disk.sda.in‚0.109375åŸH,’Ü ‚(:‡ ±ÝâÔ·â–u gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.347' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.0694 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Á´T³@Ý ‚(:µ ±ÝâÔ¾ˆ¹x disk.dm-0.out‚10439.08984375( proc.memory.percent‚17.7567720413208) disk./.usagePercent‚70.08043224230158 proc.cpu.threads‚17# disk.nvme0n1.out‚10439.08984375 disk.rbd2.out‚ 145.439453125 disk.sda2.out‚0# proc.memory.rssMB‚ 2909.26953125! disk.nvme0c0n1.in‚ 71.57421875 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.sda1.in‚0 cpu‚25.013305713386668 disk.dm-1.out‚ 75.37109375 disk.sda.in‚0.109375 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.42578125 disk.dm-0.in‚ 71.57421875) proc.memory.availableMB‚ 1855.08984375 disk.nvme0n1.in‚ 71.57421875 disk.rbd0.in‚0$ memory_percent‚88.67743015289307% disk./.usageGB‚1044.2490844726562 disk.sda3.out‚ 75.37109375 disk.sda.out‚ 75.37109375 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0% disk.nvme0c0n1.out‚10439.08984375 network.sent‚ 3.901304e+06 network.recv‚662337 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875Bk—‘Þ ‚(:† ÀÝâÔ½úÛu gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚94.002' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.8004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÍÞ°ÍCß ‚(:¸ ÀÝâÔ¹õ’x disk.rbd0.out‚0.078125 disk.sda.out‚ 75.859375( proc.memory.availableMB‚ 1855.3359375 disk.nvme0n1.in‚ 71.57421875 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0$ memory_percent‚88.67592811584473 network.recv‚666939) disk./.usagePercent‚70.08042891420364 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.out‚0( proc.memory.percent‚17.7567720413208% disk./.usageGB‚1044.2490348815918 disk.dm-1.out‚ 75.859375 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875! disk.rbd2.out‚145.58837890625& disk.nvme0n1.out‚10539.53466796875# disk.dm-0.out‚10539.53466796875 disk.sda3.in‚0.109375 disk.sda3.out‚ 75.859375 disk.drbd1777.out‚0# proc.memory.rssMB‚ 2909.26953125 proc.cpu.threads‚17 network.sent‚ 3.907385e+06 disk.sda.in‚0.109375 disk.rbd0.in‚0( disk.nvme0c0n1.out‚10539.53466796875 disk.dm-0.in‚ 71.57421875 cpu‚25.01817506989206 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.sda1.out‚0 disk.rbd3.out‚ 1.42578125,ÝSX2à - _wandbruntime‚1863  _runtime‚1863zÿÔCÇá Á _step‚163 & lr-AdamW/pg1‚0.0014359724881725072 & lr-AdamW/pg2‚0.0014359724881725072 trainer/global_step‚2049 % _timestamp‚1.7883911081106699e+09 _runtime‚1863.866669163ÃÞ*ïÌâ Æ & lr-AdamW/pg2‚0.0014359724881725072 trainer/global_step‚2049 % _timestamp‚1.7883911081106699e+09 _runtime‚1863.866669163 _step‚163 & lr-AdamW/pg1‚0.0014359724881725072£XªÊ2ã - _wandbruntime‚1863  _runtime‚1863q®·£tä ÞO GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43678497 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚21471256 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚43655899 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚78738469 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚35059972 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43655899 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43678497 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1686163122688 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35059972 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚140519969664512 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1685966943744 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1767381269504 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1767608715264 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚21471259 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35059970 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚136005935860994 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1676556264070 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚137650739012608 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚137650721973248 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚32436914 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10965658 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚139336688916992 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚140519717315072 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚138752336045568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1686163122688 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32437434 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚134329187600384 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 VHDeviceStatsMonitor.on_train_batch_suŠlptart/inactive_split_bytes.all.current‚ 252349440 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚137650721973248 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 % _timestamp‚1.7883911081118155e+09 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚139336902135296 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚139336688916992 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚138752360949248 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚137650739012608 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚139336902135296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1685966943744 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚134329170561024 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 trainer/global_step‚2049 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10966175 _step‚164 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0¯úÆ›é'å ãO XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚21471259 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35059970 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚136005935860994 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1676556264070 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚137650739012608 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚137650721973248 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚32436914 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10965658 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚139336688916992 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚140519717315072 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚138752336045568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1686163122688 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32437434 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚134329187600384 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 252349440 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚137650721973248 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 % _timestamp‚1.7883911081118155e+09 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚139336902135296 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚139336688916992 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚138752360949248 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚137650739012608 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚139336902135296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1685966943744 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚134329170561024 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 trainer/global_step‚2049 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10966175 _step‚164 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43678497 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚21471256 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚43655899 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚78738469 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚35059972 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43655899 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43678497 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1686163122688 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35059972 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚140519969664512 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1685966943744 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1767381269504 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1767608715264 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0¤ôA²2æ - _wandbruntime‚1864  _runtime‚1864. €ýî&ç èM PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚140586344266240 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43699395 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚139402739576320 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚137715761010176 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚137715761010176 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1686961526784 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚137715778049536 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚78775887 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35076492 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35076492 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32452679 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1768215708160 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚78775887 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚138817861541888 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1677350147018 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚2049 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚139402526344704 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚78753261 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚134392769945600 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 _step‚165 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43676771 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _runtime‚1864.624474178 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35076490 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1686961526784 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1686765334528 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚134392786984960 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1686765334528 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚78753261 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10971386 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚138817886445568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1768457820672 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚136070329128726 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚139402739576320 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32452118 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 % _timestamp‚1.7883911088679998e+09 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚137715778049536 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚140586077250048 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35076490 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚561 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43676771 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21481290 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1677542143766ß(2d"è íM WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35076492 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35076492 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32452679 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1768215708160 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚78775887 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚138817861541888 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1677350147018 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚2049 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚139402526344704 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚78753261 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚134392769945600 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 _step‚165 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43676771 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _runtime‚1864.624474178 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35076490 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1686961526784 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1686765334528 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚134392786984960 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1686765334528 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚78753261 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10971386 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚138817886445568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1768457820672 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚136070329128726 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚139402739576320 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32452118 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 % _timestamp‚1.7883911088679998e+09 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚137715778049536 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚140586077250048 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35076490 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚561 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43676771 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21481290 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1677542143766 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚140586344266240 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43699395 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚139402739576320 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚137715761010176 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚137715761010176 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated_•°d‚ 509607936 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1686961526784 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚137715778049536 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚78775887 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896¥¬Ì 2é - _wandbruntime‚1864  _runtime‚18642׸]ê × /loss/FRt2/detection_loss‚1.8956475669658313 /loss/FRt4/assignment_loss‚4.043462283231995 1loss/FBt4/assignment_loss‚0.13065946148292693 /loss/FRt1/detection_loss‚2.4495551827248563 % _timestamp‚1.7883911088698635e+09 _runtime‚1864.625689207 _step‚166 0loss/FRt1/assignment_loss‚3.5204501265890142 2loss/SRqqt1/assignment_loss‚1.7401080861249392 0loss/SRqqt4/detection_loss‚2.287793983731951 2loss/SRqqt2/assignment_loss‚1.4865016778792017 0loss/FBt2/assignment_loss‚0.1617291772721566 1loss/SRqqt2/detection_loss‚1.0153870804985958 /loss/FBt2/detection_loss‚0.6147859181148996 0loss/FRt3/assignment_loss‚3.8238469453418955 /loss/FBt1/detection_loss‚2.7175816322366395 trainer/global_step‚2049 1loss/SRqqt1/detection_loss‚1.9221038599224651 .loss/FBt4/detection_loss‚2.365005759256227 %loss/total_loss‚46.81448633089498 epoch‚3 0loss/FRt2/assignment_loss‚2.2182366226309087 2loss/SRqqt3/assignment_loss‚2.0334188831479927 1loss/SRqqt4/assignment_loss‚2.044361521942275 1loss/FBt1/assignment_loss‚0.12171433612032463 1loss/FBt3/assignment_loss‚0.29373221107594233 /loss/FRt3/detection_loss‚2.5860410613172196 /loss/FRt4/detection_loss‚2.3399223441427406 1loss/SRqqt3/detection_loss‚1.9911416231017365 .loss/FBt3/detection_loss‚3.011298986042247x°\-bë Ü 2loss/SRqqt2/assignment_loss‚1.4865016778792017 0loss/FBt2/assignment_loss‚0.1617291772721566 1loss/SRqqt2/detection_loss‚1.0153870804985958 /loss/FBt2/detection_loss‚0.6147859181148996 0loss/FRt3/assignment_loss‚3.8238469453418955 /loss/FBt1/detection_loss‚2.7175816322366395 trainer/global_step‚2049 1loss/SRqqt1/detection_loss‚1.9221038599224651 .loss/FBt4/detection_loss‚2.365005759256227 %loss/total_loss‚46.81448633089498 epoch‚3 0loss/FRt2/assignment_loss‚2.2182366226309087 2loss/SRqqt3/assignment_loss‚2.0334188831479927 1loss/SRqqt4/assignment_loss‚2.044361521942275 1loss/FBt1/assignment_loss‚0.12171433612032463 1loss/FBt3/assignment_loss‚0.29373221107594233 /loss/FRt3/detection_loss‚2.5860410613172196 /loss/FRt4/detection_loss‚2.3399223441427406 1loss/SRqqt3/detection_loss‚1.9911416231017365 .loss/FBt3/detection_loss‚3.011298986042247 /loss/FRt2/detection_loss‚1.8956475669658313 /loss/FRt4/assignment_loss‚4.043462283231995 1loss/FBt4/assignment_loss‚0.13065946148292693 /loss/FRt1/detection_loss‚2.4495551827248563 % _timestamp‚1.7883911088698635e+09 _runtime‚1864.625689207 _step‚166 0loss/FRt1/assignment_loss‚3.5204501265890142 2loss/SRqqt1/assignment_loss‚1.7401080861249392 0loss/SRqqt4/detection_loss‚2.287793983731951¦d£í›ì ‚(: ÏÝâÔ‹ŠÁu gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.8' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.560000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‰°ÛIí ‚(:¾ ÏÝâÔ„„åw disk.drbd1777.out‚0 network.sent‚ 3.914065e+06) disk./.usagePercent‚70.08043838648241 disk.sda.in‚0.109375 disk.sda1.in‚0 disk.rbd0.in‚0( disk.nvme0c0n1.out‚10539.98388671875! disk.nvme0c0n1.in‚ 71.57421875 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 76.26171875# proc.memory.rssMB‚ 2909.26953125 proc.cpu.threads‚17 network.recv‚671750 disk.nvme0n1.in‚ 71.57421875 disk.sda2.out‚0! disk.rbd2.out‚146.63134765625 disk.sda3.out‚ 76.26171875 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.rbd3.in‚0) proc.memory.availableMB‚ 1856.34765625 cpu‚25.022972512001257 disk.sda.out‚ 76.26171875 disk.drbd1777.in‚0# disk.dm-0.out‚10539.98388671875 disk.rbd1.out‚ 0.982421875" memory_percent‚88.669753074646( proc.memory.percent‚17.7567720413208% disk./.usageGB‚1044.2491760253906 disk.sda2.in‚0 disk.dm-0.in‚ 71.57421875& disk.nvme0n1.out‚10539.98388671875 disk.rbd2.in‚0 disk.rbd1.in‚0·×Byî ‚(:’ ÞÝâÔ¤í¿u gpu.0.gpu‚47.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚153.437' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.687400000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Fñ`uHï ‚(:½ ÞÝâÔ¬ÖÅw disk.sda3.out‚ 76.8515625 disk.rbd1.in‚0 network.sent‚ 4.000823e+06 disk.drbd1777.out‚0 disk.rbd3.in‚0# proc.memory.rssMB‚ 2909.26953125( proc.memory.percent‚17.7567720413208 proc.cpu.threads‚17 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.sda.in‚0.109375! disk.nvme0c0n1.in‚ 71.57421875 disk.dm-0.in‚ 71.57421875 disk.rbd2.in‚0) disk./.usagePercent‚70.08043838648241& disk.nvme0n1.out‚10638.92919921875 disk.sda.out‚ 76.8515625 disk.sda1.out‚0$ memory_percent‚88.66980075836182 disk.rbd1.out‚ 0.982421875 cpu‚25.027694990242644 disk.dm-1.out‚ 76.8515625! disk.rbd2.out‚147.78369140625 disk.nvme0n1.in‚ 71.57421875 disk.rbd0.in‚0( disk.nvme0c0n1.out‚10638.92919921875 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 network.recv‚677369 disk.rbd0.out‚0.078125# disk.dm-0.out‚10638.92919921875) proc.memory.availableMB‚ 1856.33984375% disk./.usageGB‚1044.2491760253906 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.in‚0Ë0hþ‘ð ‚(:† íÝâԃϧu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚98.551' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.7102 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0@60zGñ ‚(:¼ íÝâÔÇ£Áx disk.sda.in‚0.109375 disk.sda3.out‚ 77.31640625 disk.rbd2.in‚0& disk.nvme0n1.out‚10737.28076171875# proc.memory.rssMB‚ 2909.26953125 cpu‚25.03231549477341 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.rbd1.out‚ 0.982421875 disk.sda.out‚ 77.31640625 network.recv‚681158 disk.nvme0n1.in‚ 71.57421875 disk.sda1.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17' proc.memory.availableMB‚ 1856.578125 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 71.57421875 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 77.31640625 disk.rbd1.in‚0# memory_percent‚88.6683464050293( proc.memory.percent‚17.7567720413208% disk./.usageGB‚1044.2491760253906 disk.drbd1777.out‚0 disk.dm-0.in‚ 71.57421875! disk.rbd2.out‚148.88916015625 network.sent‚ 4.006254e+06) disk./.usagePercent‚70.08043838648241 disk.rbd0.in‚0( disk.nvme0c0n1.out‚10737.28076171875# disk.dm-0.out‚10737.28076171875Ñ@è2ò - _wandbruntime‚1906  _runtime‚1906Û¾YÇó Á & lr-AdamW/pg1‚0.0014173259843755445 & lr-AdamW/pg2‚0.0014173259843755445 trainer/global_step‚2099 % _timestamp‚1.7883911502830663e+09 _runtime‚1906.039890248 _step‚167¬,äºÌô Æ % _timestamp‚1.7883911502830663e+09 _runtime‚1906.039890248 _step‚167 & lr-AdamW/pg1‚0.0014173259843755445 & lr-AdamW/pg2‚0.0014173259843755445 trainer/global_step‚2099§ê*ò2õ - _wandbruntime‚1906  _runtime‚1906F–Ùä'ö ÞO LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚21972956 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚142028610861568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44702851 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚558 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚143838218976768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 250514432 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1726110490112 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1716469580018 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137509156829184 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137509139789824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 % _timestamp‚1.7883911502838106e+09 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1726110490112 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1809608115200 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚142028635765248 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35885970 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1716277583478 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚2099 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚139225417373302 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚142628588130816 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚142628588130816 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚555 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚140902690859008 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11227163 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚168 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚80588821 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44725449 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚142628801349120 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚35885970 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33200122 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚143838494394880 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚80611421 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚139225626409202 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚33199564 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚21972959 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1725914311168 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚142628801349120 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1809858629632 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44702851 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44725449 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35885972 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚140902673819648 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1725914311168 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚142628801349120 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1809858629632 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44702851 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44725449 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35885972 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚140902673819648 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44702851 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚558 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚143838218976768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 250514432 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1726110490112 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1716469580018 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137509156829184 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137509139789824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 % _timestamp‚1.7883911502838106e+09 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1726110490112 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1809608115200 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚142028635765248 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35885970 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1716277583478 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚2099 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚139225417373302 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚142628588130816 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚142628588130816 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚555 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚140902690859008 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11227163 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚168 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚80588821 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44725449 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚142628801349120 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚35885970 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33200122 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚143838494394880 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0¨häfŒ2ø - _wandbruntime‚1906  _runtime‚1906­Ž×k ù èM KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11232285 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1717263463174 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚21982993 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 _runtime‚1906.793210552 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚140967729895936 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1810457131008 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚137572739174400 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44746347 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33214777 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1726908894208 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚142094136357888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚142694638790144 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11231787 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚143904854213632 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44723723 WCDeviceStatsMonitor.on_train_batch_end/requested_byi#¥ƒtes.all.allocated‚139290019676934 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1717071466426 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21982990 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚140967712856576 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚142694638790144 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883911510363822e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35902492 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35902490 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚140967729895936 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚142094161261568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚140967712856576 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35902492 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44746347 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚498 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚80648839 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚143904593488896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1726712701952 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1726712701952 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚80626213 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚501 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚80648839 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚142694425558528 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1810692952064 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 trainer/global_step‚2099 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚44723723 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35902490 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243ŽÊ tó&ú íM UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚142694638790144 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 % _timestamp‚1.7883911510363822e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35902492 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35902490 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚140967729895936 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚142094161261568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚140967712856576 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35902492 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44746347 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚498 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚80648839 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚143904593488896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1726712701952 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1726712701952 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚80626213 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚501 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚80648839 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚142694425558528 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1810692952064 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 trainer/global_step‚2099 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚44723723 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35902490 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11232285 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1717263463174 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚21982993 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 _runtime‚1906.793210552 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚140967729895936 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1810457131008 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚137572739174400 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44746347 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33214777 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1726908894208 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚142094136357888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚142694638790144 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11231787 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚143904854213632 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44723723 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚139290019676934 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1717071466426 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21982990 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚140967712856576 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896©“ÿÆ2û - _wandbruntime‚1906  _runtime‚1906OG¿¦Yü Ó _step‚170 2loss/SRqqt1/assignment_loss‚1.8738122869105567 /loss/FBt1/detection_loss‚2.4662874000413075 1loss/SRqqt4/assignment_loss‚2.118134293036583 .loss/FRt4/detection_loss‚2.649457508534716 1loss/SRqqt3/detection_loss‚1.8122933315018477 /loss/FBt4/detection_loss‚2.9777739908960132 epoch‚3 0loss/FRt3/assignment_loss‚3.7683599167399935 1loss/FBt2/assignment_loss‚0.13109175669011475 0loss/FBt3/assignment_loss‚0.4855530677336396 .loss/FBt2/detection_loss‚0.593911697347926 /loss/FRt1/assignment_loss‚3.652531230842674 2loss/SRqqt3/assignment_loss‚1.7172136554904913 /loss/FRt3/detection_loss‚2.2978801442517174 &loss/total_loss‚45.932906095122746 trainer/global_step‚2099 /loss/FRt4/assignment_loss‚4.109355978798448 0loss/FBt4/assignment_loss‚0.4777630664822128 2loss/SRqqt2/assignment_loss‚1.3182315496181636 1loss/SRqqt1/detection_loss‚1.8595555132343655 -loss/FBt3/detection_loss‚2.25995605190595 % _timestamp‚1.7883911510385444e+09 1loss/FBt1/assignment_loss‚0.07541425266754231 /loss/FRt1/detection_loss‚2.4616381352419383 1loss/SRqqt2/detection_loss‚0.9933019055235655 0loss/SRqqt4/detection_loss‚2.257623711075538 _runtime‚ 1906.79439823 0loss/FRt2/assignment_loss‚1.9043225355770277 /loss/FRt2/detection_loss‚1.6714431149804074‡þr{^ý Ø _runtime‚ 1906.79439823 0loss/FRt2/assignment_loss‚1.9043225355770277 /loss/FRt2/detection_loss‚1.6714431149804074 _step‚170 2loss/SRqqt1/assignment_loss‚1.8738122869105567 /loss/FBt1/detection_loss‚2.4662874000413075 1loss/SRqqt4/assignment_loss‚2.118134293036583 .loss/FRt4/detection_loss‚2.649457508534716 1loss/SRqqt3/detection_loss‚1.8122933315018477 /loss/FBt4/detection_loss‚2.9777739908960132 epoch‚3 0loss/FRt3/assignment_loss‚3.7683599167399935 1loss/FBt2/assignment_loss‚0.13109175669011475 0loss/FBt3/assignment_loss‚0.4855530677336396 .loss/FBt2/detection_loss‚0.593911697347926 /loss/FRt1/assignment_loss‚3.652531230842674 2loss/SRqqt3/assignment_loss‚1.7172136554904913 /loss/FRt3/detection_loss‚2.2978801442517174 &loss/total_loss‚45.932906095122746 trainer/global_step‚2099 /loss/FRt4/assignment_loss‚4.109355978798448 0loss/FBt4/assignment_loss‚0.4777630664822128 2loss/SRqqt2/assignment_loss‚1.3182315496181636 1loss/SRqqt1/detection_loss‚1.8595555132343655 -loss/FBt3/detection_loss‚2.25995605190595 % _timestamp‚1.7883911510385444e+09 1loss/FBt1/assignment_loss‚0.07541425266754231 /loss/FRt1/detection_loss‚2.4616381352419383 1loss/SRqqt2/detection_loss‚0.9933019055235655 0loss/SRqqt4/detection_loss‚2.257623711075538ª6ÓÙ‡’þ ‚(:‡ üÝâÔͱÇu gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚35.0 gpu.0.powerWatts‚153.911' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.7822 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0•£zyIÿ ‚(:¾ üÝâÔÙ¹ïw disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 network.recv‚685473 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875 disk.sda.in‚0.109375 disk.sda1.out‚0( proc.memory.percent‚17.7567720413208 cpu‚25.036785131306154 disk.rbd0.out‚0.078125# disk.dm-0.out‚10738.22802734375( proc.memory.availableMB‚ 1856.3046875 disk.sda.out‚ 77.98828125! disk.nvme0c0n1.in‚ 71.57421875 disk.sda3.in‚0.109375) disk./.usagePercent‚70.08043838648241 disk.rbd3.in‚0 disk.sda3.out‚ 77.98828125 disk.rbd2.in‚0 disk.rbd2.out‚150.0537109375 disk.sda1.in‚0# proc.memory.rssMB‚ 2909.26953125% disk./.usageGB‚1044.2491760253906 disk.nvme0n1.in‚ 71.57421875 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ memory_percent‚88.67001533508301 network.sent‚ 4.012411e+06 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 77.98828125& disk.nvme0n1.out‚10738.22802734375 disk.rbd0.in‚0( disk.nvme0c0n1.out‚10738.22802734375 disk.dm-0.in‚ 71.57421875 proc.cpu.threads‚17!çf)€‚(:’ ‹ÞâÔíÐÞu gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.776' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.555200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0%/X@‚(:µ ‹ÞâÔœ¾±x disk.rbd2.out‚151.1630859375 disk.sda.out‚ 78.484375$ memory_percent‚88.66851329803467 disk.rbd3.in‚0( disk.nvme0c0n1.out‚10839.59130859375 disk.dm-1.out‚ 78.484375 disk.sda2.in‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 1856.55078125( proc.memory.percent‚17.7567720413208) disk./.usagePercent‚70.08062475996734 disk.dm-1.in‚0.109375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 network.recv‚691961 disk.dm-0.in‚ 71.57421875 disk.rbd1.in‚0 disk.nvme0n1.in‚ 71.57421875 disk.sda1.in‚0# proc.memory.rssMB‚ 2909.26953125 proc.cpu.threads‚17 network.sent‚ 4.099916e+06! disk./.usageGB‚1044.251953125 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375! disk.nvme0c0n1.in‚ 71.57421875# disk.dm-0.out‚10839.59130859375 disk.sda3.out‚ 78.484375 disk.rbd1.out‚ 0.982421875& disk.nvme0n1.out‚10839.59130859375 disk.sda.in‚0.109375 cpu‚25.041404949419128 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd2.in‚0[Zš[2‚- _wandbruntime‚1948  _runtime‚1948g]‰ÇƒÁ & lr-AdamW/pg2‚0.0013982135638061213 trainer/global_step‚2149 % _timestamp‚1.7883911923330042e+09 _runtime‚1948.089974774 _step‚171 & lr-AdamW/pg1‚0.0013982135638061213‰³vVÌ„Æ & lr-AdamW/pg1‚0.0013982135638061213 & lr-AdamW/pg2‚0.0013982135638061213 trainer/global_step‚2149 % _timestamp‚1.7883911923330042e+09 _runtime‚1948.089974774 _step‚171« ›ðA2…- _wandbruntime‚1948  _runtime‚1948Sá¶%†ÞO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚1948.090070494 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚33959680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚45772401 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚145920487344640 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚144154642705408 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚144154625666048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1765861678592 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1755998902886 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1851733768192 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚82461773 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33960190 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚510 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11485024 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22474656 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚144154625666048 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚142445107921510Ü’n3é'‡ãO E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚82461773 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33960190 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚510 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11485024 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22474656 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚144154625666048 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚142445107921510 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚1948.090070494 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚33959680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚45772401 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚145920487344640 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚144154642705408 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚144154625666048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1765861678592 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1755998902886 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1851733768192 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568¬"o¼?2ˆ- _wandbruntime‚1948  _runtime‚1948ÒçÙ¼î&‰èM JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚145986324772352 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1766856261632 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1766660069376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚82521791 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚36728490 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚45793299 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚551 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚45770675 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚142509501189034 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚147223248971264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚36728490 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚548 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7883911930905957e+09 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚147222969372160 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚140752725442560 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1756792785834 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚45793299 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22484693 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚33975413 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚22484690 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚45770675 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚145370436077568 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚145370411173888 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚140752708403200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1766856261632 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _step‚173 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚36728492 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚82499165 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚144219664702976 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11490172 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _runtime‚1948.847172023 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1766660069376 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33974862 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1852558198272 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1756984782582 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚144219664702976 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚145986538003968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888~oDÊó&ŠíM B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1766856261632 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _step‚173 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚36728492 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚82499165 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚144219664702976 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11490172 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _runtime‚1948.847172023 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1766660069376 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33974862 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1852558198272 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1756984782582 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚144219664702976 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚145986538003968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚145986324772352 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1766856261632 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1766660069376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚82521791 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚36728490 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚45793299 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚551 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚45770675 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚142509501189034 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚147223248971264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚36728490 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚548 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7883911930905957e+09 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚147222969372160 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚140752725442560 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1756792785834 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚45793299 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22484693 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚33975413 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚22484690 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚45770675 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚145370436077568 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚145370411173888 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚140752708403200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0­V%u2‹- _wandbruntime‚1948  _runtime‚1948 ö–ZŒÔ 1loss/SRqqt2/detection_loss‚1.1705426282607592 /loss/FRt2/detection_loss‚2.0072054735665184 /loss/FRt2/assignment_loss‚2.132328351724495 2loss/SRqqt2/assignment_loss‚1.4784856070406163 /loss/FRt4/detection_loss‚2.8374089842937034 0loss/SRqqt4/detection_loss‚2.360112157650292 % _timestamp‚1.7883911930925667e+09 _runtime‚1948.848378696 /loss/FRt4/assignment_loss‚4.353193320211817 /loss/FBt4/detection_loss‚2.0760085701942446 /loss/FBt2/detection_loss‚0.7602361765763673 %loss/total_loss‚47.73075653717416 1loss/SRqqt3/detection_loss‚1.9048367434077793 0loss/FBt4/assignment_loss‚0.2016251304477919 /loss/FRt1/detection_loss‚2.3449663170563277 .loss/FRt1/assignment_loss‚3.61961656360216 0loss/FRt3/assignment_loss‚3.6746908286949256 2loss/SRqqt3/assignment_loss‚1.8604552699459924 1loss/SRqqt4/assignment_loss‚2.316114497836679 1loss/SRqqt1/detection_loss‚1.7257423075879443 2loss/SRqqt1/assignment_loss‚1.8323143659127425 /loss/FBt3/detection_loss‚2.8909577429294586 epoch‚3 _step‚174 1loss/FBt1/assignment_loss‚0.31819527223706245 0loss/FBt2/assignment_loss‚0.1502362621563178 0loss/FBt3/assignment_loss‚0.2624977861802715 /loss/FBt1/detection_loss‚2.8228751212358474 trainer/global_step‚2149 /loss/FRt3/detection_loss‚2.6301110584240455®‰_“ìÙ /loss/FBt2/detection_loss‚0.7602361765763673 %loss/total_loss‚47.73075653717416 1loss/SRqqt3/detection_loss‚1.9048367434077793 0loss/FBt4/assignment_loss‚0.2016251304477919 /loss/FRt1/detection_loss‚2.3449663170563277 .loss/FRt1/assignment_loss‚3.61961656360216 0loss/FRt3/assignment_loss‚3.6746908286949256 2loss/SRqqt3/assignment_loss‚1.8604552699459924 1loss/SRqqt4/assignment_loss‚2.316114497836679 1loss/SRqqt1/detection_loss‚1.725742307587-[Ä2s9443 2loss/SRqqt1/assignment_loss‚1.8323143659127425 /loss/FBt3/detection_loss‚2.8909577429294586 epoch‚3 _step‚174 1loss/FBt1/assignment_loss‚0.31819527223706245 0loss/FBt2/assignment_loss‚0.1502362621563178 0loss/FBt3/assignment_loss‚0.2624977861802715 /loss/FBt1/detection_loss‚2.8228751212358474 trainer/global_step‚2149 /loss/FRt3/detection_loss‚2.6301110584240455 1loss/SRqqt2/detection_loss‚1.1705426282607592 /loss/FRt2/detection_loss‚2.0072054735665184 /loss/FRt2/assignment_loss‚2.132328351724495 2loss/SRqqt2/assignment_loss‚1.4784856070406163 /loss/FRt4/detection_loss‚2.8374089842937034 0loss/SRqqt4/detection_loss‚2.360112157650292 % _timestamp‚1.7883911930925667e+09 _runtime‚1948.848378696 /loss/FRt4/assignment_loss‚4.353193320211817 /loss/FBt4/detection_loss‚2.0760085701942446®Œ2’Ž‚(:‡ šÞâÔòô÷v gpu.0.gpu‚28.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚112.734' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.5468 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0í®>EC‚(:¸ šÞâԆűx disk.drbd1777.out‚0 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.sda1.in‚0 disk.sda3.out‚ 78.796875 cpu‚25.045970204522686 disk.nvme0n1.in‚ 71.57421875& disk.nvme0n1.out‚10939.03076171875! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.in‚0) proc.memory.availableMB‚ 1856.32421875 network.sent‚ 4.105141e+06 disk.rbd0.in‚0# disk.dm-0.out‚10939.03076171875( proc.memory.percent‚17.7567720413208 network.recv‚695776% disk./.usageGB‚1044.2520942687988 disk.rbd1.out‚ 0.982421875 disk.rbd0.out‚0.078125 disk.dm-0.in‚ 71.57421875 disk.sda.in‚0.109375 disk.drbd1777.in‚0 disk.rbd3.out‚ 1.44140625$ memory_percent‚88.66989612579346# proc.memory.rssMB‚ 2909.26953125 proc.cpu.threads‚17( disk./.usagePercent‚70.0806342322461 disk.rbd2.out‚153.3896484375 disk.sda.out‚ 78.796875( disk.nvme0c0n1.out‚10939.03076171875 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.dm-1.out‚ 78.796875 disk.rbd1.in‚0 disk.sda1.out‚0ì Œ³‚(:„ ©ÞâԜѰu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.06' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.812 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0/°ÛC‘‚(:¸ ©ÞâÔ•¼öw proc.cpu.threads‚17 disk.rbd2.in‚0% disk.nvme0n1.out‚10939.4931640625' proc.memory.availableMB‚ 1856.484375 network.sent‚ 4.192008e+06 disk.sda.out‚ 79.55078125" disk.dm-0.out‚10939.4931640625$ memory_percent‚88.66891860961914( proc.memory.percent‚17.7567720413208 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0.109375 disk.sda.in‚0.109375! disk.nvme0c0n1.in‚ 71.57421875 disk.rbd3.in‚0 disk.dm-1.out‚ 79.55078125( disk./.usagePercent‚70.0806342322461 disk.sda1.out‚0' disk.nvme0c0n1.out‚10939.4931640625 disk.rbd3.out‚ 1.44140625# proc.memory.rssMB‚ 2909.26953125 network.recv‚700709 disk.sda1.in‚0 disk.dm-0.in‚ 71.57421875 disk.rbd1.in‚0 disk.rbd1.out‚ 0.982421875 disk.rbd0.in‚0 disk.sda3.out‚ 79.55078125% disk./.usageGB‚1044.2520942687988 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.dm-1.in‚0.109375 disk.rbd2.out‚154.4404296875 disk.nvme0n1.in‚ 71.57421875 cpu‚25.05023935347967¦1c’‚(:… ¸ÞâÔ¼Û¨u gpu.0.gpu‚32.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.405' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.881 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ÿþ?7H“‚(:½ ¸ÞâÔÂÒÏx# disk.dm-0.out‚11038.44482421875 disk.rbd1.out‚ 0.982421875 disk.sda.out‚ 79.87890625 disk.rbd0.in‚0 network.sent‚ 4.198213e+06( disk./.usagePercent‚70.0806342322461 disk.rbd1.in‚0 disk.drbd1777.in‚0$ memory_percent‚88.66894245147705# proc.memory.rssMB‚ 2909.26953125 cpu‚25.05453808285498 disk.rbd3.in‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.dm-1.out‚ 79.87890625 disk.rbd2.out‚155.6123046875( disk.nvme0c0n1.out‚11038.44482421875 disk.sda2.out‚0 network.recv‚705621 disk.dm-0.in‚ 71.57421875 disk.sda3.out‚ 79.87890625& disk.nvme0n1.out‚11038.44482421875 disk.sda1.in‚0 disk.drbd1777.out‚0( proc.memory.percent‚17.7567720413208 proc.cpu.threads‚17 disk.nvme0n1.in‚ 71.57421875 disk.dm-1.in‚0.109375! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.in‚0 disk.rbd2.in‚0 disk.rbd0.out‚0.078125) proc.memory.availableMB‚ 1856.48046875% disk./.usageGB‚1044.2520942687988 disk.sda.in‚0.109375 disk.sda1.out‚0PÆhØ2”- _wandbruntime‚1990  _runtime‚1990rö¤\Å•¿ % lr-AdamW/pg1‚0.001378651184018577 % lr-AdamW/pg2‚0.001378651184018577 trainer/global_step‚2199 % _timestamp‚1.7883912344409835e+09 _runtime‚1990.196960666 _step‚175I¿OºÊ–Ä % lr-AdamW/pg1‚0.001378651184018577 % lr-AdamW/pg2‚0.001378651184018577 trainer/global_step‚2199 % _timestamp‚1.7883912344409835e+09 _runtime‚1990.196960666 _step‚175¯$±ñ’2—- _wandbruntime‚1990  _runtime‚1990דЛã'˜ÝO =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚528 _step‚176 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1805809046016 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚46819353 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚149212386558464 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚147406577512448 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚46819353 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚ 1990.20200455 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚149212386558464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚149212599776768 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚37537970 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚46796755 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚2199 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22976356 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚148581160493568 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚150475216014848 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚147406577512448 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚84357325 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚143869095286784 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1893784297472 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚145664798469718 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1805809046016 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚149212386558464 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚147406577512448 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚46819353 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚ 1990.20200455 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚149212386558464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚149212599776768 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚37537970 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚46796755 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚2199 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22976356 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚148581160493568 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚150475216014848 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚147406577512448 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚84357325 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚143869095286784 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1893784297472 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚145664798469718 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1805809046016 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚84372117 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚84394743 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚143932694671360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1894622805504 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚149278437217792 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22986393 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚148646685989888 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 $ _timestamp‚1.788391235190063e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚37554492 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1894867015168 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1806607436800 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚145729191737242 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚147471633588736 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1796514105242 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚150541577908736 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚34736309 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚148646710893568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _step‚177 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚149278223986176 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1806803629056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚46840251 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚143932677632000 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚1990.946887391 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚37554490 OBDeviceStatsMoniK6tò8tor.on_train_batch_end/inactive_split.all.allocated‚34736823 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚46840251 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1796706101990 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚511 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚37554492 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚147471633588736 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚147471616549376 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚149278437217792 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚147471616549376 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚514 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚145729400773350G¿ò&œìM >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚84372117 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚84394743 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚143932694671360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1894622805504 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚149278437217792 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22986393 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚148646685989888 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 $ _timestamp‚1.788391235190063e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚37554492 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1894867015168 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1806607436800 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚145729191737242 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚147471633588736 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1796514105242 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚150541577908736 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚34736309 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚148646710893568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _step‚177 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚149278223986176 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1806803629056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚46840251 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚143932677632000 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚1990.946887391 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚37554490 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚34736823 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚46840251 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1796706101990 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚511 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚37554492 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚147471633588736 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚147471616549376 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚149278437217792 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚147471616549376 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚514 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚145729400773350±]„×ö2- _wandbruntime‚1990  _runtime‚1990ŒBD[žÕ /loss/FRt1/detection_loss‚2.2323425041109908 /loss/FBt1/detection_loss‚2.8118454109538686 .loss/FBt4/detection_loss‚2.901043470089252 1loss/SRqqt2/detection_loss‚0.9598500274949604 /loss/FBt2/detection_loss‚0.6587761218509366 0loss/FRt4/assignment_loss‚3.9994110616048175 1loss/SRqqt2/assignment_loss‚1.394790179696348 0loss/FBt2/assignment_loss‚0.1856783462375072 $ _timestamp‚1.788391235192379e+09 _runtime‚1990.948138689 0loss/FRt2/assignment_loss‚2.1623176691623835 /loss/FRt3/detection_loss‚2.6202663392634005 /loss/FBt3/detection_loss‚2.3370561599731445 _step‚178 /loss/FRt1/assignment_loss‚3.665204673421149 2loss/SRqqt3/assignment_loss‚1.9854977939191252 /loss/FRt2/detection_loss‚1.8608359384278839 2loss/SRqqt4/assignment_loss‚2.3011776731361318 1loss/FBt1/assignment_loss‚0.28540295758020046 1loss/FBt3/assignment_loss‚0.33005047612823546 1loss/SRqqt3/detection_loss‚1.9003710077945577 1loss/SRqqt4/detection_loss‚2.1257044710690463 /loss/FRt4/detection_loss‚2.5097340762615206 %loss/total_loss‚46.48319464864177 epoch‚3 /loss/FRt3/assignment_loss‚3.410422146320343 1loss/FBt4/assignment_loss‚0.40983589048174995 /loss/SRqqt1/detection_loss‚1.75337496223086 trainer/global_step‚2199 2loss/SRqqt1/assignment_loss‚1.68220529143335457—–`ŸÚ 2loss/SRqqt1/assignment_loss‚1.6822052914333545 /loss/FRt1/detection_loss‚2.2323425041109908 /loss/FBt1/detection_loss‚2.8118454109538686 .loss/FBt4/detection_loss‚2.901043470089252 1loss/SRqqt2/detection_loss‚0.9598500274949604 /loss/FBt2/detection_loss‚0.6587761218509366 0loss/FRt4/assignment_loss‚3.9994110616048175 1loss/SRqqt2/assignment_loss‚1.394790179696348 0loss/FBt2/assignment_loss‚0.1856783462375072 $ _timestamp‚1.788391235192379e+09 _runtime‚1990.948138689 0loss/FRt2/assignment_loss‚2.1623176691623835 /loss/FRt3/detection_loss‚2.6202663392634005 /loss/FBt3/detection_loss‚2.3370561599731445 _step‚178 /loss/FRt1/assignment_loss‚3.665204673421149 2loss/SRqqt3/assignment_loss‚1.9854977939191252 /loss/FRt2/detection_loss‚1.8608359384278839 2loss/SRqqt4/assignment_loss‚2.3011776731361318 1loss/FBt1/assignment_loss‚0.28540295758020046 1loss/FBt3/assignment_loss‚0.33005047612823546 1loss/SRqqt3/detection_loss‚1.9003710077945577 1loss/SRqqt4/detection_loss‚2.1257044710690463 /loss/FRt4/detection_loss‚2.5097340762615206 %loss/total_loss‚46.48319464864177 epoch‚3 /loss/FRt3/assignment_loss‚3.410422146320343 1loss/FBt4/assignment_loss‚0.40983589048174995 /loss/SRqqt1/detection_loss‚1.75337496223086 trainer/global_step‚2199²ó7Ã’ ‚(:‡ ÇÞâÔèàÊu gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.433' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.4866 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0èÂãE¡‚(:º ÇÞâÔ®è¢x( proc.memory.availableMB‚ 1856.1484375 disk.rbd3.out‚ 1.44140625 disk.sda.in‚0.109375 disk.rbd1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.rbd2.in‚0 disk.drbd1777.in‚0 network.sent‚ 4.204762e+06 network.recv‚710277 disk.rbd2.out‚157.7373046875 disk.nvme0n1.in‚ 71.57421875( disk.nvme0c0n1.out‚11038.80419921875 cpu‚25.05880126586669 disk.dm-0.in‚ 71.57421875# proc.memory.rssMB‚ 2909.26953125( proc.memory.percent‚17.7567720413208 disk.rbd3.in‚0# disk.dm-0.out‚11038.80419921875 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 80.5234375 disk.rbd0.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 80.5234375% disk./.usageGB‚1044.2525253295898& disk.nvme0n1.out‚11038.80419921875 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 71.57421875 disk.sda.out‚ 80.5234375 disk.rbd1.out‚ 0.982421875 disk.sda1.in‚0$ memory_percent‚88.67096900939941 proc.cpu.threads‚17) disk./.usagePercent‚70.08066316109746 disk.sda3.in‚0.109375OéÊ¢‚(:’ ÖÞâÔÒˆŸu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚138.669' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.733800000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0H¡(J£‚(:¿ ÖÞâÔ•ù“y# proc.memory.rssMB‚ 2909.26953125 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚11137.29248046875( proc.memory.percent‚17.7567720413208% disk./.usageGB‚1044.2524528503418 disk.rbd3.in‚0 disk.sda3.out‚ 81.04296875 network.sent‚ 4.291686e+06 disk.drbd1777.in‚0 disk.dm-0.in‚ 71.57421875 disk.sda3.in‚0.109375 disk.dm-1.out‚ 81.04296875 network.recv‚715504 disk.sda.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd1.out‚ 0.982421875 disk.rbd2.out‚158.8896484375& disk.nvme0n1.out‚11137.29248046875) proc.memory.availableMB‚ 1855.83984375 proc.cpu.threads‚17 disk.sda.out‚ 81.04296875 disk.sda2.out‚0# disk.dm-0.out‚11137.29248046875 disk.rbd2.in‚0$ memory_percent‚88.67285251617432 disk.rbd3.out‚ 1.44140625 cpu‚25.063092051787358) disk./.usagePercent‚70.08065829695433 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.in‚0 disk.rbd1.in‚0 disk.nvme0n1.in‚ 71.57421875”ý‰·’¤‚(:‡ åÞâÔݳÑu gpu.0.gpu‚37.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.673' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.9346 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‚Æâ|?¥‚(:´ åÞâÔâ½³x( disk.nvme0c0n1.out‚11235.71533203125( proc.memory.percent‚17.7567720413208 disk.sda3.in‚0.109375 disk.rbd2.in‚0 disk.nvme0n1.in‚ 71.57421875 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 71.57421875 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 71.57421875 network.recv‚718932 disk.dm-1.out‚ 81.765625 disk.rbd0.out‚0.078125 disk.rbd3.in‚0# memory_percent‚88.6728286743164& disk.nvme0n1.out‚11235.71533203125 disk.sda.in‚0.109375# proc.memory.rssMB‚ 2909.26953125) disk./.usagePercent‚70.08063576829132 disk.sda3.out‚ 81.765625 disk.sda1.out‚0 disk.drbd1777.out‚0# disk.dm-0.out‚11235.71533203125 disk.rbd2.out‚159.9990234375 disk.sda1.in‚0 disk.drbd1777.in‚0& proc.memory.availableMB‚ 1855.84375 cpu‚25.06736208212832 proc.cpu.threads‚17 disk.rbd3.out‚ 1.44140625 network.sent‚ 4.297039e+06 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.rbd1.out‚ 1.130859375 disk.sda.out‚ 81.765625% disk./.usageGB‚1044.2521171569824Läëµ2¦- _wandbruntime‚2032  _runtime‚2032HöæǧÁ & lr-AdamW/pg1‚0.0013586551782521958 & lr-AdamW/pg2‚0.0013586551782521958 trainer/global_step‚2249 % _timestamp‚1.7883912764038534e+09 _runtime‚2032.159833861 _step‚179áÊ6Ì¨Æ % _timestamp‚1.7883912764038534e+09 _runtime‚2032.159833861 _step‚179 & lr-AdamW/pg1‚0.0013586551782521958 & lr-AdamW/pg2‚0.0013586551782521958 trainer/global_step‚2249³©Hf2©- _wandbruntime‚2032  _runtime‚2032Ð%³ä'ªÞO UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚150658529358848 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚23478059 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚86230277 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚38363970 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚152504285772288 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231640064 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256543744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚152504498990592 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 % _timestamp‚1.7883912764049077e+09 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚47843707 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚153793425063424 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚147049064515584 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1845952592384 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚152504285772288 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚151857435309568 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚38363972 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1845952592384 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12002650 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚150658529358848 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚47866305 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1835633538242 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚505 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚153793168519680 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚151857460213248 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1835441541702 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚148884489017926 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚502 _runtime‚2032.161131736 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚47843707 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚148884698053826 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚150658546398208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚38363972 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1935964850176 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1935733210112 trainer/global_step‚2249 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚35481211 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12003152 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚147049047476224 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚38363970 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚180 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚86207677 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1845756413440 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚35480706 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚47866305¿»¿#«ãO VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚148884489017926 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚502 _runtime‚2032.161131736 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚47843707 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚148884698053826 ]IDeviceStatsMonitor.on_train_batch_start/active_bytesäÎK–á#.large_pool.allocated‚150658546398208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚38363972 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1935964850176 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1935733210112 trainer/global_step‚2249 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚35481211 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12003152 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚147049047476224 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚38363970 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚180 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚86207677 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1845756413440 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚35480706 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚47866305 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚150658529358848 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚23478059 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚86230277 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚38363970 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚152504285772288 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231640064 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 256543744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚152504498990592 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 % _timestamp‚1.7883912764049077e+09 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚47843707 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚153793425063424 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚147049064515584 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1845952592384 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚152504285772288 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚151857435309568 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚38363972 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1845952592384 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12002650 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚150658529358848 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚47866305 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1835633538242 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚505 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚153793168519680 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚151857460213248 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1835441541702´5ÑÍÑ2¬- _wandbruntime‚2032  _runtime‚2032L*óí&­çM >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚86267695 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚47864579 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚47887203 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚86267695 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚150723568395776 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚38380490 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1936811745792 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 _step‚181 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚47887203 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1846750996480 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚38380492 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚150723585435136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1846554804224 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚152570336431616 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚86245069 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚147112646860800 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1936559147520 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚148949091321558 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚148948882285450 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚153859519953408 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788391277158892e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚38380492 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1846750996480 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚153859797455360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1846554804224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277501952 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23488090 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚35495733 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚151922985709568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚86245069 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252598272 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0z¬ù}ò&®ìM VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12008166 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚2249 _runtime‚2032.915297108 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚38380490 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚86267695 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚47864579 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚47887203 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚86267695 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚150723568395776 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚38380490 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1936811745792 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 _step‚181 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚47887203 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1846750996480 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚38380492 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚150723585435136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1846554804224 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚152570336431616 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚86245069 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚147112646860800 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1936559147520 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚148949091321558 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚148948882285450 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚153859519953408 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788391277158892e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚38380492 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1846750996480 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚153859797455360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1846554804224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277501952 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23488090 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚35495733 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚151922985709568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚86245069 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252598272 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 PDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1885703780864 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚48890659 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155796184986112 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚88080629 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚153910481205248 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚152104388602034 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1885703780864 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚157111583498752 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1977621023744 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36240422 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚511 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1977848469504 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1885899959808 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚23979756 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚155133710125568 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚155796184986112 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚153910498244608 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚152104179566134 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚39189970 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1885899959808 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚88103229 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36239908 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 _step‚184 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155796398204416 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚153910498244608 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚39189972 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12260152 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1885703780864 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚48890659 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155796184986112 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚88080629 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚153910481205248 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚152104388602034 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1885703780864 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚157111583498752 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1977621023744 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36240422 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚511 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1977848469504 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1885899959808 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚23979756 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚155133710125568 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚155796184986112 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚153910498244608 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚152104179566134 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚39189970 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1885899959808 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚88103229 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36239908 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 _step‚184 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155796398204416 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚153910498244608 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚39189972 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12260152 IDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚155862235645440 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚23989793 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23989790 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚36255602 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1978466894336 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚152168572833658 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚155862022413824 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚48911531 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 % _timestamp‚1.7883913190032566e+09 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1886502171648 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚157177967435264 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1886502171648 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚153975520242176 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚88140647 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚153975537281536 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚150292633128960 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚2299 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚152168781869766 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚39206490 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚155862235645440 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚153975537281536 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚157177702516224 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚36255050 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚155199260525568 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚88118021 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚39206492 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚153975520242176 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚150292616089600 \JDeviceStatsMonitor.on_tá³ïrain_batch_end/requested_bytes.small_pool.allocated‚ 1876148740806 _step‚185 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚88140647 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12265809 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚48911531 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚48934155 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚39206490 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚155862235645440 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚549 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚153975537281536 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚157177702516224 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚36255050 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚155199260525568 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚88118021 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚39206492 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚153975520242176 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚150292616089600 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1876148740806 _step‚185 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚88140647 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12265809 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚48911531 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚48934155 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚155862235645440 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚23989793 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23989790 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚36255602 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1978466894336 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚152168572833658 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚155862022413824 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065¹d6Q2Á- _wandbruntime‚2074  _runtime‚2074d¨pO[ÂÕ .loss/FRt3/detection_loss‚2.576192793654807 %loss/total_loss‚46.86623937661392 /loss/FRt1/assignment_loss‚3.327495400678544 1loss/FBt2/assignment_loss‚0.18075149675283078 1loss/FBt4/assignment_loss‚0.27129813736925523 /loss/FBt4/detection_loss‚2.4146593610445657 0loss/FRt2/assignment_loss‚1.8994083593387414 .loss/FRt1/detection_loss‚2.310062260003317 1loss/SRqqt1/detection_loss‚1.7237279535341663 0loss/SRqqt3/detection_loss‚2.021445076516334 /loss/FBt1/detection_loss‚2.8831004053354263 % _timestamp‚1.7883913190048833e+09 _step‚186 .loss/FRt4/assignment_loss‚4.27607135956104 2loss/SRqqt3/assignment_loss‚2.1148284461269986 2loss/SRqqt4/assignment_loss‚2.2387529271795894 0loss/SRqqt2/detection_loss‚1.173094539369783 /loss/FBt2/detection_loss‚0.7693211842192019 _runtime‚2074.760647745 2loss/SRqqt2/assignment_loss‚1.4710140710808017 1loss/FBt3/assignment_loss‚0.25957651583191294 /loss/FRt2/detection_loss‚1.7770519035287422 /loss/FRt4/detection_loss‚2.3467782974243163 trainer/global_step‚2299 epoch‚3 1loss/FBt1/assignment_loss‚0.13524538327328628 1loss/SRqqt4/detection_loss‚2.3305610183513528 /loss/FBt3/detection_loss‚2.7368936446996837 /loss/FRt3/assignment_loss‚3.784386879132118 2loss/SRqqt1/assignment_loss‚1.8445219626071072ÒäÀW`ÃÚ 2loss/SRqqt1/assignment_loss‚1.8445219626071072 .loss/FRt3/detection_loss‚2.576192793654807 %loss/total_loss‚46.86623937661392 /loss/FRt1/assignment_loss‚3.327495400678544 1loss/FBt2/assignment_loss‚0.18075149675283078 1loss/FBt4/assignment_loss‚0.27129813736925523 /loss/FBt4/detection_loss‚2.4146593610445657 0loss/FRt2/assignment_loss‚1.8994083593387414 .loss/FRt1/detection_loss‚2.310062260003317 1loss/SRqqt1/detection_loss‚1.7237279535341663 0loss/SRqqt3/detection_loss‚2.021445076516334 /loss/FBt1/detection_loss‚2.8831004053354263 % _timestamp‚1.7883913190048833e+09 _step‚186 .loss/FRt4/assignment_loss‚4.27607135956104 2loss/SRqqt3/assignment_loss‚2.1148284461269986 2loss/SRqqt4/assignment_loss‚2.2387529271795894 0loss/SRqqt2/detection_loss‚1.173094539369783 /loss/FBt2/detection_loss‚0.7693211842192019 _runtime‚2074.760647745 2loss/SRqqt2/assignment_loss‚1.4710140710808017 1loss/FBt3/assignment_loss‚0.25957651583191294 /loss/FRt2/detection_loss‚1.7770519035287422 /loss/FRt4/detection_loss‚2.3467782974243163 trainer/global_step‚2299 epoch‚3 1loss/FBt1/assignment_loss‚0.13524538327328628 1loss/SRqqt4/detection_loss‚2.3305610183513528 /loss/FBt3/detection_loss‚2.7368936446996837 /loss/FRt3/assignment_loss‚3.784386879132118ºÄOÐKÄ‚(:’ ¡ßâÔãä¦u gpu.0.gpu‚31.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚155.397' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.079399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0í ëEÅ‚(:º ¡ßâÔ¯”Àw disk.sda.in‚0.109375 disk.sda.out‚ 84.16015625 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.dm-1.out‚ 84.16015625 disk.rbd2.in‚0! disk.dm-0.out‚11434.310546875 disk.rbd1.in‚0 disk.rbd1.out‚ 1.22607421875) proc.memory.percent‚17.75679588317871 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.44140625) disk./.usagePercent‚70.08062808806528 disk.rbd0.in‚0& disk.nvme0c0n1.out‚11434.310546875 disk.dm-0.in‚ 71.57421875 disk.sda3.out‚ 84.16015625 disk.nvme0n1.in‚ 71.57421875$ memory_percent‚88.67514133453369 network.sent‚ 4.40207e+06 disk.dm-1.in‚0.109375! disk.nvme0c0n1.in‚ 71.57421875 disk.rbd2.out‚168.6123046875 network.recv‚737192% disk./.usageGB‚1044.2520027160645 disk.sda1.out‚0 disk.sda2.out‚0$ disk.nvme0n1.out‚11434.310546875) proc.memory.availableMB‚ 1855.46484375" proc.memory.rssMB‚ 2909.2734375 cpu‚25.083464906611454 disk.rbd0.out‚0.078125 disk.sda3.in‚0.109375 proc.cpu.threads‚17JwÅ’Æ‚(:‡ °ßâÔêêÏu gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.791' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.1582 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æüÚMÇ‚(: °ßâÔë‚Çx network.sent‚ 4.489501e+06) disk./.usagePercent‚70.08048753992898 disk.dm-1.in‚0.109375 disk.sda.out‚ 84.97265625 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 71.58203125 disk.dm-1.out‚ 84.97265625 disk.rbd1.in‚0 disk.nvme0n1.in‚ 71.58203125& disk.nvme0n1.out‚11536.55810546875( disk.nvme0c0n1.out‚11536.55810546875! disk.rbd2.out‚170.90576171875 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.44140625 cpu‚25.087408764078425 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125 disk.sda2.in‚0# disk.dm-0.out‚11536.55810546875 disk.sda3.out‚ 84.97265625" proc.memory.rssMB‚ 2909.2734375 disk.sda1.in‚0 disk.dm-0.in‚ 71.58203125) proc.memory.availableMB‚ 1855.81640625 disk.sda1.out‚0 disk.sda2.out‚0 disk.rbd1.out‚ 1.22607421875$ memory_percent‚88.67299556732178 network.recv‚743614% disk./.usageGB‚1044.2499084472656 disk.rbd0.in‚0 disk.sda3.in‚0.109375) proc.memory.percent‚17.75679588317871 proc.cpu.threads‚17­‹ÇÈ‚(:’ ¿ßâÔÂÔÀu gpu.0.gpu‚36.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚35.0 gpu.0.powerWatts‚155.041' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.008199999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0k›Ãæ3É‚(:¨ ¿ßâԚŵx) proc.memory.availableMB‚ 1855.46484375) proc.memory.percent‚17.75679588317871 disk.rbd1.out‚ 1.2890625" proc.memory.rssMB‚ 2909.2734375 cpu‚25.091213984707547 disk.drbd1777.out‚0) disk./.usagePercent‚70.08053797341323 disk.nvme0c0n1.in‚74.25! disk.rbd2.out‚173.11669921875 network.sent‚ 4.495997e+06 network.recv‚748263 disk.sda2.out‚0" disk.dm-0.out‚11635.8154296875 disk.drbd1777.in‚0% disk.nvme0n1.out‚11635.8154296875 disk.sda1.out‚0' disk.nvme0c0n1.out‚11635.8154296875 disk.sda3.in‚0.109375 disk.sda3.out‚ 85.71484375 disk.dm-1.out‚ 85.71484375 disk.rbd2.in‚0 disk.nvme0n1.in‚74.25 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚74.25$ memory_percent‚88.67514133453369 proc.cpu.threads‚17 disk.rbd0.out‚0.078125 disk.rbd3.out‚ 1.44140625 disk.rbd1.in‚0 disk.rbd0.in‚0$ disk./.usageGB‚1044.250659942627 disk.sda.in‚0.109375 disk.sda.out‚ 85.71484375 disk.dm-1.in‚0.109375i¤¥2Ê- _wandbruntime‚2115  _runtime‚2115ƒmãÆËÀ trainer/global_step‚2349 $ _timestamp‚1.788391359793387e+09 _runtime‚2115.549354974 _step‚187 & lr-AdamW/pg1‚0.0013174294180397228 & lr-AdamW/pg2‚0.0013174294180397228Ê%ÆËÌÅ $ _timestamp‚1.788391359793387e+09 _runtime‚2115.549354974 _step‚187 & lr-AdamW/pg1‚0.0013174294180397228 & lr-AdamW/pg2‚0.0013174294180397228 trainer/global_step‚2349»’eÏ2Í- _wandbruntime‚2115  _runtime‚2115"3jã'ÎÝO WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2019737719808 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12517228 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚158410009845248 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚153408985933824 _step‚188 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1925651148288 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2019973554176 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚155324079150242 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚89953581 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚2349 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚89976181 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1914884180518 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚157162450091008 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36998687 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚157162433051648 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚49960209 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚158409984941568 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚49937611 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1915076177058 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚40015970 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12516711 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚157162433051648 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚159088084199936 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1925651148288 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36998167 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚155323870114342 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚49960209 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚157162450091008 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24481459 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1925847327232 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚49937611 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚159088084199936 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1925847327232 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚520 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚40015972 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚159088297418240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚160429722661376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40015972 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24481456 $ _timestamp‚1.788391359794462e+09&Ý*Q ÏâO IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12517228 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚158410009845248 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚153408985933824 _step‚188 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1925651148288 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚517 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2019973554176 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚155324079150242 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚89953581 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚2349 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚89976181 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1914884180518 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.l%¹‡a—arge_pool.allocated‚157162450091008 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36998687 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚157162433051648 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚49960209 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚158409984941568 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚49937611 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1915076177058 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚40015970 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12516711 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚157162433051648 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚159088084199936 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1925651148288 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36998167 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚155323870114342 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚49960209 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚157162450091008 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24481459 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1925847327232 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚49937611 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚159088084199936 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1925847327232 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚520 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚40015972 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚159088297418240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚160429722661376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40015972 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24481456 $ _timestamp‚1.788391359794462e+09 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2019737719808 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824¼QœÔ2Ð- _wandbruntime‚2116  _runtime‚2116$ôàî&ÑèM D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2116.299252809 _step‚189 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1926449539072 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚40032490 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚153472602357760 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1915678063466 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚37013819 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚24491490 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2020568244736 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚157227489127936 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚160496078682624 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2020808260096 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚49981107 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚512 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚160496343601664 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37013307 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚49981107 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚157227472088576 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚89990973 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚509 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚159153921627648 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚49958483 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚90013599 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚2349 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚159154134859264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0·–w…ó&ÒíM OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚37013819 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚24491490 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2020568244736 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚157227489127936 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚160496078682624 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2020808260096 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚49981107 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚512 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚160496343601664 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37013307 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚49981107 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚157227472088576 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚89990973 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚509 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚159153921627648 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚49958483 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚90013599 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚2349 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚159154134859264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2116.299252809 _step‚189 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1926449539072 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚40032490 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚153472602357760 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1915678063466 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0½i&ž2Ó- _wandbruntime‚2116  _runtime‚21167õBYÔÓ /loss/FBt3/detection_loss‚2.3417477673954434 0loss/FRt1/assignment_loss‚3.3846174092083188 2loss/SRqqt2/assignment_loss‚1.3706311986818909 1loss/FBt1/assignment_loss‚0.27064173373702105 /loss/FRt3/detection_loss‚2.4057465129813482 .loss/FRt4/detection_loss‚2.341910472346677 _runtime‚2116.300591481 _step‚190 0loss/FRt2/assignment_loss‚1.9823195951007238 /loss/FRt4/assignment_loss‚4.512276114688979 1loss/SRqqt3/assignment_loss‚2.000732925194113 1loss/FBt3/assignment_loss‚0.23945444263517857 0loss/FBt4/assignment_loss‚0.3015184124550698 /loss/FBt2/detection_loss‚0.4146922698710114 -loss/FBt4/detection_loss‚2.85150761495937 trainer/global_step‚2349 0loss/FRt3/assignment_loss‚3.5006564259529114 epoch‚3 0loss/SRqqt1/assignment_loss‚1.70184861766089 2loss/SRqqt4/assignment_loss‚2.1356601995318685 1loss/SRqqt3/detection_loss‚2.1248878778322884 % _timestamp‚1.7883913605447283e+09 /loss/FRt2/detection_loss‚1.8089967580320678 1loss/SRqqt2/detection_loss‚1.2600105149149894 /loss/FBt1/detection_loss‚2.7128663460413613 0loss/FBt2/assignment_loss‚0.1250219154577391 /loss/FRt1/detection_loss‚2.3526155339492547 1loss/SRqqt1/detection_loss‚1.7713268016456465 1loss/SRqqt4/detection_loss‚2.3820104776923334 $loss/total_loss‚46.2936979379665…Z›^ÕØ % _timestamp‚1.7883913605447283e+09 /loss/FRt2/detection_loss‚1.8089967580320678 1loss/SRqqt2/detection_loss‚1.2600105149149894 /loss/FBt1/detection_loss‚2.7128663460413613 0loss/FBt2/assignment_loss‚0.1250219154577391 /loss/FRt1/detection_loss‚2.3526155339492547 1loss/SRqqt1/detection_loss‚1.7713268016456465 1loss/SRqqt4/detection_loss‚2.3820104776923334 $loss/total_loss‚46.2936979379665 /loss/FBt3/detection_loss‚2.3417477673954434 0loss/FRt1/assignment_loss‚3.3846174092083188 2loss/SRqqt2/assignment_loss‚1.3706311986818909 1loss/FBt1/assignment_loss‚0.27064173373702105 /loss/FRt3/detection_loss‚2.4057465129813482 .loss/FRt4/detection_loss‚2.341910472346677 _runtime‚2116.300591481 _step‚190 0loss/FRt2/assignment_loss‚1.9823195951007238 /loss/FRt4/assignment_loss‚4.512276114688979 1loss/SRqqt3/assignment_loss‚2.000732925194113 1loss/FBt3/assignment_loss‚0.23945444263517857 0loss/FBt4/assignment_loss‚0.3015184124550698 /loss/FBt2/detection_loss‚0.4146922698710114 -loss/FBt4/detection_loss‚2.85150761495937 trainer/global_step‚2349 0loss/FRt3/assignment_loss‚3.5006564259529114 epoch‚3 0loss/SRqqt1/assignment_loss‚1.70184861766089 2loss/SRqqt4/assignment_loss‚2.1356601995318685 1loss/SRqqt3/detection_loss‚2.1248878778322884¾£œåÖ‚(:’ ÎßâÔ“€©u gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.961' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.992200000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0#‡%ö7ׂ(:¬ ÎßâÔˆîÈx$ memory_percent‚88.67387771606445) proc.memory.percent‚17.75679588317871( disk./.usagePercent‚70.0805594780461& disk.nvme0c0n1.out‚11637.572265625 disk.sda1.out‚0 disk.rbd3.in‚0 network.recv‚752098 disk.drbd1777.out‚0 disk.sda3.out‚ 86.265625 disk.rbd1.out‚ 1.2890625$ disk.nvme0n1.out‚11637.572265625 proc.cpu.threads‚17 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 86.265625 disk.nvme0n1.in‚ 74.3515625" proc.memory.rssMB‚ 2909.2734375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125! disk.dm-0.out‚11637.572265625 disk.rbd2.in‚0 disk.sda.out‚ 86.265625 disk.rbd3.out‚ 1.44140625 network.sent‚ 4.501287e+06 disk.drbd1777.in‚0 disk.rbd1.in‚0 cpu‚25.095073922987925 disk.nvme0c0n1.in‚ 74.3515625 disk.dm-0.in‚ 74.3515625 disk.sda.in‚0.109375 disk.sda1.in‚0' proc.memory.availableMB‚ 1855.671875% disk./.usageGB‚1044.2509803771973 disk.sda2.in‚0 disk.sda3.in‚0.109375! disk.rbd2.out‚176.40185546875Àk†Ü›Ø‚(: ÝßâÔÌŸâu gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.47' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.093999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*+]6±Ù‚(:¯ ÝßâÔÞ àx! disk.nvme0c0n1.in‚ 75.28515625 disk.rbd3.out‚ 1.44140625& proc.memory.availableMB‚ 1855.90625' disk.nvme0c0n1.out‚11738.3291015625 disk.rbd1.in‚0% disk.nvme0n1.out‚11738.3291015625 disk.sda1.out‚0) proc.memory.percent‚17.75679588317871 network.recv‚757341% disk./.usageGB‚1044.2503242492676" disk.dm-0.out‚11738.3291015625 disk.sda3.out‚87.46875 disk.rbd皉0.in‚0 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚88.67244720458984 network.sent‚ 4.588071e+06 disk.drbd1777.in‚0 disk.drbd1777.out‚0 cpu‚25.098658736705815 disk.rbd2.in‚0 disk.sda.out‚87.46875" proc.memory.rssMB‚ 2909.2734375 disk.rbd0.out‚0.078125 disk.dm-1.in‚0.109375 disk.dm-1.out‚87.46875 disk.rbd1.out‚ 1.2890625! disk.rbd2.out‚178.55029296875 disk.sda.in‚0.109375 disk.sda1.in‚0 proc.cpu.threads‚17 disk.dm-0.in‚ 75.28515625 disk.sda3.in‚0.109375 disk.nvme0n1.in‚ 75.28515625 disk.rbd3.in‚0) disk./.usagePercent‚70.08051544475022 ®OÔ2Ú- _wandbruntime‚2157  _runtime‚2157+{ð\ÇÛÁ & lr-AdamW/pg1‚0.0012962340842629736 & lr-AdamW/pg2‚0.0012962340842629736 trainer/global_step‚2399 % _timestamp‚1.7883914019505618e+09 _runtime‚2157.706522202 _step‚191ÿZiÌÜÆ _runtime‚2157.706522202 _step‚191 & lr-AdamW/pg1‚0.0012962340842629736 & lr-AdamW/pg2‚0.0012962340842629736 trainer/global_step‚2399 % _timestamp‚1.7883914019505618e+09¿ðo%Î2Ý- _wandbruntime‚2157  _runtime‚2157˺C–ã'ÞÝO ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1965598515712 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚161686284661248 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2062019511296 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚158543560662550 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚37757779 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚163748304172544 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚2157.709998483 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚161686259757568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1954797496466 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1954605499926 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _step‚192 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚51007161 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24983159 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚156588955162624 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24983156 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254446592 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚162379983413760 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚160414384898048 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚163748049725952 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚162380196632064 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚156588972201984 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚91849133 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40841972 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚160414401937408 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚40841970 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚160414401937408Ìi]è'ßâO IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚91849133 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40841972 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚160414401937408 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚40841970 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚160414401937408 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1965598515712 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚161686284661248 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2062019511296 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚158543560662550 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚37757779 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚163748304172544 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚2157.709998483 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚161686259757568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1954797496466 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1954605499926 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _step‚192 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚51007161 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24983159 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚156588955162624 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24983156 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254446592 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚162379983413760 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚160414384898048 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚163748049725952 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚162380196632064 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚156588972201984À”ø.2à- _wandbruntime‚2158  _runtime‚2158lfõí&áçM D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12780206 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚162445820841472 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚91863925 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚160479423934976 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1955591379622 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚40858492 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚156652554547200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚160479440974336 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37772869 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚160479423934976 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚24993190 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚40858492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚2158.475445708 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚158608162966182 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚51028059 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚160479440974336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚40858490 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚156652571586560 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚91863925 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1966593098752 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚91886551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0Eï¹PâìM RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1966396906496 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚162446034073088 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚51028059 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2062625246208 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚163814410500096 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚40858490 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚91886551 J=DeviceStatsMonitor.on_train_batch_e½Ž¢"nd/active.small_pool.freed‚51005435 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚158607953930074 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚2399 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚161751785253888 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252598272 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1966593098752 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _step‚193 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚161751810157568 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚160479423934976 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1955591379622 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚40858492 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚156652554547200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚160479440974336 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37772869 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚160479423934976 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚24993190 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚40858492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚2158.475445708 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚158608162966182 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚51028059 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚160479440974336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚40858490 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚156652571586560 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚91863925 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1966593098752 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚91886551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12780206 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚162445820841472 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚91863925 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 P&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3728848397731781 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.12014631181955338 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.53125 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.46628686785697937 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.24494951963424683 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.46666666865348816 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3600747883319855 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3444444537162781 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39180052280426025 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3580247461795807 particle/accuracy_5_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3384774625301361 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.32837197184562683 particle/accuracy_3_of_4‚1 particle/accuracy_5_of_5‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.37142854928970337 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.27259209752082825 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.34375 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.00853301864117384 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.24654383957386017 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.09345238655805588 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10499681532382965 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2916666865348816 ,"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.375 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 _runtime‚2217.705340627 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9928185939788818 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9522003531455994 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.28880682587623596 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 particle/accuracy_1_of_3‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 jet/accuracy_1_of_7‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2291552871465683 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_6‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.986423671245575 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08168771117925644 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.12136977165937424 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09519554674625397 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 6validation_average_jet_accuracy‚0.3652424216270447 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09259258955717087 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.14156557619571686 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 particle/accuracy_3_of_6‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.12666666507720947 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.40151792764663696 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.992301881313324 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3862837255001068 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9970614910125732 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.14651162922382355 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 (particle/accuracy‚0.8473201990127563 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.009780408814549446 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11124339699745178 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,jet/accuracy_3_of_6‚0.017543859779834747 particle/accuracy_2_of_6‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.285714328289032 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.967129647731781 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 particle/accuracy_2_of_2‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10794451087713242 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16655400395393372 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3334101140499115 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9912575483322144 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3193880319595337 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9979093670845032 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.28214287757873535 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.32477959990501404 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12077295035123825 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9914921522140503 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.31398192048072815 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.298042356967926 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1449395716190338 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07586207240819931 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 *loss/val_total_loss‚46.993983546600965 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3750787079334259 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.32007330656051636 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4198427200317383 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.32722222805023193 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.15262766182422638 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3144878149032593 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9784946441650391 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.02538382261991501 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.2297234982252121 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.11036888509988785 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 6 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.287804514169693 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.014016897417604923 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1111111119389534 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5158185362815857 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.009744873270392418 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_7_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.36403509974479675 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.28696128726005554 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6491227746009827 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.24409523606300354 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.16319534182548523 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1111111119389534 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 jet/accuracy_5_of_5‚0 particle/accuracy_5_of_7‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.13182209432125092 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 jet/accuracy_2_of_7‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48145127296447754 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.16538077592849731 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17954705655574799 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07144840806722641 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1270001381635666 9"Purity/2FRt2SRqqt1FBt/SRqqt_puritü²¨T`Xy‚0.3791666626930237 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9905203580856323 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005811056587845087 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.16199439764022827 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.32003486156463623 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9200000762939453 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.02611367218196392 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9850037097930908 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.49444445967674255 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11574074625968933 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.26923078298568726 jet/accuracy_4_of_7‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03804127126932144 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6315789222717285 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11682843416929245 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3488647937774658 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9844884276390076 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +jet/accuracy_3_of_5‚0.08338802307844162 particle/accuracy_4_of_5‚1 (particle/f_score‚0.36474257707595825 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.025227969512343407 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9583333134651184 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.04555555805563927 +jet/accuracy_2_of_3‚0.19341154396533966 +jet/accuracy_2_of_4‚0.27039963006973267 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3748299479484558 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9979774951934814 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.33483800292015076 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10935913771390915 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3717948794364929 trainer/global_step‚2415 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9668718576431274 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 6 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.529411792755127 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2423127144575119 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4766005277633667 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.865079402923584 epoch‚3 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.28292739391326904 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.025370370596647263 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.3655019700527191 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.285714328289032 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8180887699127197 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.2969600558280945 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_6‚0.017543859779834747 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2916666567325592 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3600747883319855 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4442176818847656 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888657093048 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.16239912807941437 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 particle/accuracy_4_of_7‚1 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.919642984867096 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910033345222473 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.00992334634065628 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 jet/accuracy_5_of_6‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.23333334922790527 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.29850995540618896 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.39539170265197754 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.31064024567604065 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3398260474205017 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4316677451133728 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 particle/accuracy_2_of_4‚1 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2514880895614624 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.13071894645690918 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.38933414220809937 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4069311022758484 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.28774121403694153 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6708306074142456 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23279687762260437 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33465608954429626 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3125000298023224 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9681547284126282 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_3‚0.01581110619008541 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.33419331908226013 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +particle/sensitivity‚0.2807129919528961 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3333677053451538 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13342857360839844 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.26522108912467957 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9906639456748962 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.12384792417287827 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.29891306161880493 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_4_of_4‚1 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4368634819984436 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4642857015132904 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.38428494334220886 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2648148238658905 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7142857313156128 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.28333333134651184 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1730687916278839 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9854142665863037 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.14091993868350983 *jet/accuracy_1_of_4‚0.8058868646621704 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7142857313156128 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34045329689979553 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8930042386054993 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10876892507076263 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2467171549797058 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.7638726234436035 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9953237175941467 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1203402653336525 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36173903942108154 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12082656472921371 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6341327428817749 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.28547152876853943 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.017204301431775093 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.062428541481494904 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3517020344734192 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.32366007566452026 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02227342501282692 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16772741079330444 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 8"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.427880197763443 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08746405690908432 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.31445595622062683 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.20884773135185242 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 jet/accuracy_5_of_7‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2632802128791809 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666716337204 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09175381064414978 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 particle/accuracy_4_of_6‚1 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3527122139930725 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.30826249718666077 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34583333134651184 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.014179370366036892 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.41255223751068115 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3818740248680115 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20093518495559692 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2981548309326172 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4515513479709625 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_7_of_7‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.003953526262193918 +jet/accuracy_3_of_4‚0.04395605996251106 particle/accuracy_1_of_2‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.17444293200969696 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2667084336280823 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2707478106021881 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.12916667759418488 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.28098076581954956 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2712121605873108 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 +jet/accuracy_2_of_2‚0.08807362616062164 particle/accuracy_2_of_7‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4437962472438812 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.12046574056148529 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3520737290382385 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 _step‚195 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27311262488365173 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.16636167466640472 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.311481773853302 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08593832701444626 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 *jet/accuracy_1_of_1‚0.4018077850341797 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 )jet/accuracy_1_of_5‚0.896425187587738 particle/accuracy_1_of_5‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 % _timestamp‚1.7883914619474635e+09 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9890960454940796 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.3994506299495697 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.1659865528345108 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.6385778784751892 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_6_of_7‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9911826848983765 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05798933282494545 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.30723366141319275 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08593832701444626 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.019288979470729828 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9963043928146362 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.1501082330942154 +jet/accuracy_2_of_6‚0.45614030957221985 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9855695366859436 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15682758390903473 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_3‚1 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9080460667610168 -jet/accuracy_4_of_4‚0.0011123470030725002 jet/accuracy_3_of_7‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75üo¬’’'ðä¥ :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07586207240819931 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 *loss/val_total_loss‚46.993983546600965 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3750787079334259 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.32007330656051636 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4198427200317383 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.32722222805023193 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.15262766182422638 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3144878149032593 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9784946441650391 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.02538382261991501 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.2297234982252121 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.11036888509988785 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 6 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.287804514169693 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.014016897417604923 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1111111119389534 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5158185362815857 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.009744873270392418 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_7_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.36403509974479675 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.28696128726005554 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6491227746009827 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.24409523606300354 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.16319534182548523 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1111111119389534 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 jet/accuracy_5_of_5‚0 particle/accuracy_5_of_7‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.13182209432125092 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 jet/accuracy_2_of_7‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48145127296447754 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.16538077592849731 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17954705655574799 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07144840806722641 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1270001381635666 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3791666626930237 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9905203580856323 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005811056587845087 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.16199439764022827 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.32003486156463623 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9200000762939453 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.02611367218196392 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9850037097930908 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.49444445967674255 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11574074625968933 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.26923078298568726 jet/accuracy_4_of_7‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03804127126932144 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6315789222717285 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11682843416929245 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3488647937774658 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9844884276390076 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +jet/accuracy_3_of_5‚0.08338802307844162 particle/accuracy_4_of_5‚1 (particle/f_score‚0.36474257707595825 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.025227969512343407 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9583333134651184 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.04555555805563927 +jet/accuracy_2_of_3‚0.19341154396533966 +jet/accuracy_2_of_4‚0.27039963006973267 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3748299479484558 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9979774951934814 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.33483800292015076 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10935913771390915 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3717948794364929 trainer/global_step‚2415 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9668718576431274 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 6 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.529411792755127 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2423127144575119 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4766005277633667 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.865079402923584 epoch‚3 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.28292739391326904 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.025370370596647263 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.3655019700527191 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.285714328289032 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8180887699127197 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.2969600558280945 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_6‚0.017543859779834747 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2916666567325592 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FRt_purity“6A£Yk‚1 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3600747883319855 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4442176818847656 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888657093048 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.16239912807941437 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 particle/accuracy_4_of_7‚1 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.919642984867096 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910033345222473 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.00992334634065628 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 jet/accuracy_5_of_6‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.23333334922790527 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.29850995540618896 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.39539170265197754 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.31064024567604065 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3398260474205017 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4316677451133728 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 particle/accuracy_2_of_4‚1 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2514880895614624 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.13071894645690918 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.38933414220809937 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4069311022758484 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.28774121403694153 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6708306074142456 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23279687762260437 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33465608954429626 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3125000298023224 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9681547284126282 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_3‚0.01581110619008541 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.33419331908226013 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +particle/sensitivity‚0.2807129919528961 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3333677053451538 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13342857360839844 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.26522108912467957 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9906639456748962 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.12384792417287827 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.29891306161880493 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_4_of_4‚1 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4368634819984436 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4642857015132904 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.38428494334220886 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2648148238658905 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7142857313156128 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.28333333134651184 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1730687916278839 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9854142665863037 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.14091993868350983 *jet/accuracy_1_of_4‚0.8058868646621704 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7142857313156128 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34045329689979553 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8930042386054993 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10876892507076263 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2467171549797058 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.7638726234436035 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9953237175941467 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1203402653336525 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36173903942108154 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12082656472921371 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6341327428817749 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.28547152876853943 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.017204301431775093 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.062428541481494904 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3517020344734192 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.32366007566452026 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02227342501282692 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16772741079330444 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 8"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.427880197763443 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08746405690908432 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.31445595622062683 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.20884773135185242 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 jet/accuracy_5_of_7‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2632802128791809 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666716337204 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09175381064414978 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 particle/accuracy_4_of_6‚1 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3527122139930725 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.30826249718666077 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34583333134651184 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.014179370366036892 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.41255223751068115 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3818740248680115 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20093518495559692 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2981548309326172 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4515513479709625 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_7_of_7‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.003953526262193918 +jet/accuracy_3_of_4‚0.04395605996251106 particle/accuracy_1_of_2‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.17444293200969696 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2667084336280823 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2707478106021881 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.12916667759418488 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.28098076581954956 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2712121605873108 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 +jet/accuracy_2_of_2‚0.08807362616062164 particle/accuracy_2_of_7‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4437962472438812 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.12046574056148529 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3520737290382385 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 _step‚195 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27311262488365173 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.16636167466640472 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.311481773853302 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08593832701444626 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 *jet/accuracy_1_of_1‚0.4018077850341797 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 )jet/accuracy_1_of_5‚0.896425187587738 particle/accuracy_1_of_5‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 % _timestamp‚1.7883914619474635e+09 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9890960454940796 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.3994506299495697 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.1659865528345108 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.6385778784751892 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_6_of_7‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9911826848983765 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05798933282494545 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.30723366141319275 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08593832701444626 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.019288979470729828 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9963043928146362 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.1501082330942154 +jet/accuracy_2_of_6‚0.45614030957221985 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9855695366859436 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15682758390903473 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_3‚1 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9080460667610168 -jet/accuracy_4_of_4‚0.0011123470030725002 jet/accuracy_3_of_7‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9982501864433289 jet/accuracy_6_of_6‚0 ) Purity/1FRt2SRqqt2FBt/FRt_purity‚0.25 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.18704745173454285 6 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.986838698387146 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.16632108390331268 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3283713459968567 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3728848397731781 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.12014631181955338 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.53125 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.46628686785697937 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.24494951963424683 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.46666666865348816 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3600747883319855 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3444444537162781 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39180052280426025 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3580247461795807 particle/accuracy_5_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3384774625301361 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.32837197184562683 particle/accuracy_3_of_4‚1 particle/accuracy_5_of_5‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.37142854928970337 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.27259209752082825 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.34375 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.00853301864117384 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.24654383957386017 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.09345238655805588 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10499681532382965 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2916666865348816 ,"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.375 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 _runtime‚2217.705340627 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9928185939788818 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9522003531455994 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.28880682587623596 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 particle/accuracy_1_of_3‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 jet/accuracy_1_of_7‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2291552871465683 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_6‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.986423671245575 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08168771117925644 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.12136977165937424 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09519554674625397 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 6validation_average_jet_accuracy‚0.3652424216270447 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09259258955717087 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.14156557619571686 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 particle/accuracy_3_of_6‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.12666666507720947 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.40151792764663696 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.992301881313324 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3862837255001068 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9970614910125732 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.14651162922382355 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 (particle/accuracy‚0.8473201990127563 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.009780408814549446 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11124339699745178 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,jet/accuracy_3_of_6‚0.017543859779834747 particle/accuracy_2_of_6‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.285714328289032 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.967129647731781 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 particle/accuracy_2_of_2‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10794451087713242 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16655400395393372 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3334101140499115 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9912575483322144 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3193880319595337 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9979093670845032 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.28214287757873535 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.32477959990501404 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12077295035123825 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9914921522140503 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.31398192048072815 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.298042356967926 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1449395716190338 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1Ãf]Y?›ñ‚(: ¨àâÔ´ç€v gpu.0.gpu‚6.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚29.0 gpu.0.powerWatts‚75.956' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚15.191200000000002 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\¢^ë8ò‚(:­ ¨àâÔ¶è½x disk.sda3.in‚0.109375# disk.nvme0n1.out‚12041.44140625 disk.sda.out‚ 90.7421875 disk.sda2.out‚0 proc.cpu.threads‚17 network.sent‚ 4.699238e+06! disk.rbd2.out‚192.44091796875 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125% disk.nvme0c0n1.out‚12041.44140625 cpu‚25.225300776886293 network.recv‚780405 disk.dm-0.out‚12041.44140625 disk.sda3.out‚ 90.7421875 disk.rbd1.out‚ 1.2890625 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0" proc.memory.rssMB‚ 2919.8515625* proc.memory.percent‚17.821359634399414) disk./.usagePercent‚70.08073330716186 disk.nvme0c0n1.in‚ 80.015625) proc.memory.availableMB‚ 1847.37109375 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.sda.in‚0.109375 disk.dm-0.in‚ 80.015625 disk.dm-1.out‚ 90.7421875 disk.nvme0n1.in‚ 80.015625 disk.sda1.in‚0 disk.drbd1777.out‚0$ memory_percent‚88.72454166412354% disk./.usageGB‚1044.2535705566406 disk.rbd2.in‚0 disk.rbd3.out‚ 1.44140625ûgÆùó‚(:… ·àâÔÐÔ¶u gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.405' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.881 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ýž2F3ô‚(:¨ ·àâÔ‰¨êw disk.sda2.in‚0) proc.memory.percent‚17.64359474182129 cpu‚25.228038753437133% disk./.usageGB‚1044.2539520263672 disk.sda.out‚ 91.390625 disk.drbd1777.in‚0 disk.drbd1777.out‚0! disk.rbd2.out‚196.76904296875( proc.memory.availableMB‚ 1874.5703125 proc.cpu.threads‚17 disk.sda1.out‚0$ memory_percent‚88.55853080749512 disk.rbd0.out‚0.078125% disk.nvme0c0n1.out‚12283.02734375 disk.dm-0.in‚ 80.734375 disk.dm-1.in‚0.109375 disk.rbd1.out‚ 1.2890625 disk.sda.in‚0.109375 disk.sda2.out‚0 disk.sda3.out‚ 91.390625# disk.nvme0n1.out‚12283.02734375 disk.dm-0.out‚12283.02734375 disk.rbd2.in‚0) disk./.usagePercent‚70.08075890791528 disk.rbd3.out‚ 1.44140625 network.sent‚ 4.802703e+06 disk.nvme0n1.in‚ 80.734375 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.109375" proc.memory.rssMB‚ 2890.7265625 disk.dm-1.out‚ 91.390625 disk.rbd1.in‚0 network.recv‚787017 disk.nvme0c0n1.in‚ 80.734375#pÔ2õ- _wandbruntime‚2248  _runtime‚2248\û͆ÆöÀ % _timestamp‚1.7883914926937912e+09 _runtime‚ 2248.45101287 _step‚196 & lr-AdamW/pg1‚0.0012746739371071853 & lr-AdamW/pg2‚0.0012746739371071853 trainer/global_step‚2449é`-Ë÷Å & lr-AdamW/pg1‚0.0012746739371071853 & lr-AdamW/pg2‚0.0012746739371071853 trainer/global_step‚2449 % _timestamp‚1.7883914926937912e+09 _runtime‚ 2248.45101287 _step‚196Ä¡„_ª2ø- _wandbruntime‚2248  _runtime‚2248“Ä ÖùÝO UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚167119841327104 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚531 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2021721937920 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚165098315568128 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚165098332607488 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚163089025147008 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚94340522 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2021525758976 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42062972 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚42062974 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚52277550 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚25782828 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2010450873084 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2010258876544 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚166428784219648 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _step‚197 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚52277550 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2021525758976 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚161078783309824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚167119841327104 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚38939872 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13157044 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚168548716945920 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2120172754944 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 $ _timestamp‚1.788391492694568e+09 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚42062972 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚163089234182908 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚161078766270464 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2119932726272 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _runtime‚2248.451144739 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚42062974 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚166428809123328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 trainer/global_step‚2449 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚165098315568128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚25782831 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3ˆKìÁè'úâO YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚166428784219648 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _step‚197 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚52277550 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2021525758976 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚161078783309824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚167119841327104 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚38939872 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13157044 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚168548716945920 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2120172754944 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 $ _timestamp‚1.788391492694568e+09 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚42062972 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚163089234182908 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚161078766270464 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2119932726272 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _runtime‚2248.451144739 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚42062974 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚166428809123328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 trainer/global_step‚2449 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚165098315568128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚25782831 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚167119841327104 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚531 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2021721937920 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚165098315568128 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚165098332607488 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚163089025147008 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚94340522 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2021525758976 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42062972 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚42062974 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚52277550 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚25782828 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2010450873084 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2010258876544Åø¦ò2û- _wandbruntime‚2249  _runtime‚2249rèn\î&üèM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165163371644416 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚52298422 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚2449 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2022324149760 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚38955389 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚165163354605056 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38954863 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚52321046 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2120777990656 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2022520342016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚94400540 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2011244756240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13162001 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚94377914 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚163153418414532 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42079492 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2121020103168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚167185678754816 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚168615354722816 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 _runtime‚2249.206110732 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚165163354605056 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13162524 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚163153627450640 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2011052759492 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚42079492 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2022324149760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚25792865sQLEÈýíM XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2022324149760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚25792865 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165163371644416 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚52298422 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚2449 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2022324149760 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚38955389 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚165163354605056 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38954863 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚52321046 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2120777990656 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2022520342016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚94400540 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2011244756240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13162001 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚94377914 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚163153418414532 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42079492 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2121020103168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚167185678754816 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚168615354722816 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 _runtime‚2249.206110732 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚165163354605056 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13162524 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚163153627450640 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2011052759492 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚42079492 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0Æ™a -2þ- _wandbruntime‚2249  _runtime‚2249pýYÿÓ /loss/FRt1/assignment_loss‚3.420501647851406 /loss/FRt3/detection_loss‚2.5100607717192016 /loss/FBt2/detection_loss‚0.5379883114796351 /loss/FBt3/detection_loss‚2.6725553975385776 2loss/SRqqt4/assignment_loss‚2.1897770410430604 .loss/FBt1/detection_loss‚2.877099527253045 0loss/FRt2/assignment_loss‚2.1475441261093215 /loss/FBt3/assignment_loss‚0.223771902877883 /loss/FRt2/detection_loss‚1.6341993930463743 epoch‚4 1loss/FBt2/assignment_loss‚0.19073097182389748 1loss/FBt1/assignment_loss‚0.19943055367265414 1loss/SRqqt2/detection_loss‚0.7181559944685721 1loss/SRqqt3/detection_loss‚2.0398259275099813 trainer/global_step‚2449 .loss/FBt4/detection_loss‚3.554859682917595 .loss/FRt3/assignment_loss‚3.30619520645637 1loss/SRqqt1/assignment_loss‚1.772910171270109 2loss/SRqqt3/assignment_loss‚1.9436777923475295 1loss/FBt4/assignment_loss‚0.17765239753352943 0loss/SRqqt1/detection_loss‚1.887902870774269 1loss/SRqqt4/detection_loss‚2.2429839325123937 _step‚199 .loss/FRt4/detection_loss‚2.379951522156999 % _timestamp‚1.7883914934514046e+09 _runtime‚2249.207220664 /loss/FRt1/detection_loss‚2.4998372388191714 /loss/FRt4/assignment_loss‚4.541246949015437 2loss/SRqqt2/assignment_loss‚1.4546962919527826 &loss/total_loss‚47.123555622149794T²}Ü^€Ø _runtime‚2249.207220664 /loss/FRt1/detection_loss‚2.4998372388191714 /loss/FRt4/assignment_loss‚4.541246949015437 2loss/SRqqt2/assignment_loss‚1.4546962919527826 &loss/total_loss‚47.123555622149794 /loss/FRt1/assignment_loss‚3.420501647851406 /loss/FRt3/detection_loss‚2.5100607717192016 /loss/FBt2/detection_loss‚0.5379883114796351 /loss/FBt3/detection_loss‚2.6725553975385776 2loss/SRqqt4/assignment_loss‚2.1897770410430604 .loss/FBt1/detection_loss‚2.877099527253045 0loss/FRt2/assignment_loss‚2.1475441261093215 /loss/FBt3/assignment_loss‚0.223771902877883 /loss/FRt2/detection_loss‚1.6341993930463743 epoch‚4 1loss/FBt2/assignment_loss‚0.19073097182389748 1loss/FBt1/assignment_loss‚0.19943055367265414 1loss/SRqqt2/detection_loss‚0.7181559944685721 1loss/SRqqt3/detection_loss‚2.0398259275099813 trainer/global_step‚2449 .loss/FBt4/detection_loss‚3.554859682917595 .loss/FRt3/assignment_loss‚3.30619520645637 1loss/SRqqt1/assignment_loss‚1.772910171270109 2loss/SRqqt3/assignment_loss‚1.9436777923475295 1loss/FBt4/assignment_loss‚0.17765239753352943 0loss/SRqqt1/detection_loss‚1.887902870774269 1loss/SRqqt4/detection_loss‚2.2429839325123937 _step‚199 .loss/FRt4/detection_loss‚2.379951522156999 % _timestamp‚1.7883914934514046e+09ǹÁt‚(:… ÆàâÔÄ©Ñu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚149.56' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.912 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚07Ç*æ"‚‚(:— ÆàâÔìýÄx disk.rbd1.in‚0 disk.sda.out‚92 disk.sda2.in‚0$ memory_percent‚88.56172561645508 proc.cpu.threads‚17 network.recv‚790685 disk.drbd1777.in‚0 disk.drbd1777.out‚0 network.sent‚ 4.807979e+06 disk.sda3.in‚0.109375 disk.dm-1.out‚92 disk.sda.in‚0.109375 disk.sda1.in‚0' proc.memory.availableMB‚ 1874.046875 disk.rbd3.out‚ 1.44140625' disk.nvme0c0n1.out‚12427.3974609375 disk.sda2.out‚0" proc.memory.rssMB‚ 2890.7265625) disk./.usagePercent‚70.08070028218994 disk.rbd3.in‚0 disk.dm-0.in‚ 80.734375 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.nvme0n1.in‚ 80.734375% disk.nvme0n1.out‚12427.3974609375 disk.rbd0.out‚0.078125 disk.rbd1.out‚ 1.2890625 disk.nvme0c0n1.in‚ 80.734375) proc.memory.percent‚17.64359474182129% disk./.usageGB‚1044.2530784606934! disk.rbd2.out‚200.00341796875 disk.sda1.out‚0 cpu‚25.23060506152672" disk.dm-0.out‚12427.3974609375 disk.sda3.out‚92 disk.rbd0.in‚0_zTƒ‚(:’ ÕàâÔ몺u gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.611' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.722199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0O4ÄDB„‚(:· ÕàâÔû²ùw disk.dm-0.in‚ 80.87890625 disk.rbd2.in‚0" proc.memory.rssMB‚ 2890.7265625 network.sent‚ 4.893556e+06% disk./.usageGB‚1044.2533645629883 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 80.87890625 disk.sda3.in‚0.109375 disk.sda3.out‚ 92.80859375 network.recv‚794914 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 92.80859375' disk.nvme0c0n1.out‚12525.9345703125) proc.memory.percent‚17.64359474182129 disk.drbd1777.out‚0" disk.dm-0.out‚12525.9345703125% disk.nvme0n1.out‚12525.9345703125 disk.sda.out‚ 92.80859375 disk.sda1.out‚0 disk.rbd3.out‚ 1.44140625! disk.rbd2.out‚204.34326171875 disk.sda.in‚0.109375$ memory_percent‚88.55905532836914 proc.cpu.threads‚17' disk./.usagePercent‚70.080719482755 disk.rbd1.in‚0 disk.sda1.in‚0 disk.rbd0.out‚0.078125' proc.memory.availableMB‚ 1874.484375 disk.rbd1.out‚ 1.2890625 disk.nvme0n1.in‚ 80.87890625 disk.rbd0.in‚0 cpu‚25.233273237364493 disk.rbd3.in‚0t®õO…‚(:… äàâÔíü­u gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.26' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.052 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“Y1ôF†‚(:» äàâÔ“å‡y% disk./.usageGB‚1044.2532501220703 disk.rbd3.in‚0 disk.rbd0.out‚0.078125 cpu‚25.235870810961295 disk.dm-1.in‚0.109375 disk.rbd1.in‚0' disk.nvme0c0n1.out‚12624.5458984375) disk./.usagePercent‚70.08071180252897 disk.sda.in‚0.109375 disk.drbd1777.in‚0) proc.memory.availableMB‚ 1874.24609375" disk.dm-0.out‚12624.5458984375 disk.dm-1.out‚ 93.15234375$ memory_percent‚88.56050968170166" proc.memory.rssMB‚ 2890.7265625 proc.cpu.threads‚17 disk.sda3.in‚0.109375 disk.rbd1.out‚ 1.2890625 disk.rbd2.in‚0 disk.nvme0n1.in‚ 80.87890625 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 80.87890625) proc.memory.percent‚17.64359474182129 disk.rbd3.out‚ 1.44140625 disk.sda.out‚ 93.15234375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.dm-0.in‚ 80.87890625! disk.rbd2.out‚209.72607421875 disk.sda1.in‚0 disk.sda2.out‚0 network.sent‚ 4.899724e+06 network.recv‚799520 disk.sda3.out‚ 93.15234375% disk.nvme0n1.out‚12624.5458984375 disk.sda2.in‚0÷sÁß2‡- _wandbruntime‚2290  _runtime‚2290]¢„LjÁ & lr-AdamW/pg1‚0.0012527669778097496 & lr-AdamW/pg2‚0.0012527669778097496 trainer/global_step‚2499 % _timestamp‚1.7883915345946586e+09 _runtime‚2290.350648815 _step‚200èÞ.xÌ‰Æ & lr-AdamW/pg1‚0.0012527669778097496 & lr-AdamW/pg2‚0.0012527669778097496 trainer/global_step‚2499 % _timestamp‚1.7883915345946586e+09 _runtime‚2290.350648815 _step‚200Èu‡¡2Š- _wandbruntime‚2290  _runtime‚2290â³ã?ã'‹ÝO TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2050172192492 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚170411740540928 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚164258752538624 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚166308715695216 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚168350267414528 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚53324502 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚171867414250496 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚169705083939328 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚96236074 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _step‚201 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚170411953759232 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚53347100 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚42888974 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚171867143026688 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788391534595711e+09 _runtime‚2290.354009072 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2061473126400 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚39700153 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42888972 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚168350284453888 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚170411953759232 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2061473126400 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚170411740540928 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚168350267414528 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚42888972 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2061669305344 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚169705059035648 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚39699617 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚53324502 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚164258735499264 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚96213474 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26284528 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13415622 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚53324502 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚164258735499264 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚96213474 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26284528 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13415622 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚170411740540928 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚164258752538624 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚166308715695216 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚168350267414528 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚53324502 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚171867414250496 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚169705083939328 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚96236074 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 _step‚201 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚170411953759232 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚53347100 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚42888974 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚171867143026688 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitoŒY"k­r.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788391534595711e+09 _runtime‚2290.354009072 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2061473126400 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚39700153 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42888972 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚168350284453888 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚170411953759232 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750É^M‡©2- _wandbruntime‚2291  _runtime‚2291å=òÍî&ŽèM K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚168415323490816 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2062467709440 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚96250866 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚39714921 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚53345374 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚39715428 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚42905494 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚170477577968640 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _runtime‚2291.113893922 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2062271517184 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚166373317998848 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚504 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚169770609435648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚168415323490816 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2062467709440 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚171933507731456 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚96273492 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2162923199488 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2050774078900 % _timestamp‚1.7883915353574831e+09 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚166373108962740 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚2499 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13420359 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚42905492 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2050966075648 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚96273492 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13420863 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚169770584531968 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚164322351923200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚53367998 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚53345374 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42905492 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚168415306451456 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚170477791200256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2062271517184 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚26294565 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0šâ!ó&íM REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚42905494 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚170477577968640 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _runtime‚2291.113893922 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2062271517184 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚166373317998848 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚504 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚169770609435648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚168415323490816 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2062467709440 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚171933507731456 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚96273492 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2162923199488 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2050774078900 % _timestamp‚1.7883915353574831e+09 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚166373108962740 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚2499 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13420359 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚42905492 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2050966075648 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚96273492 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13420863 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚169770584531968 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚164322351923200 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚53367998 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚53345374 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42905492 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚168415306451456 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚170477791200256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2062271517184 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚26294565 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚168415323490816 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2062467709440 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚96250866 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚39714921 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚53345374 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚39715428 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0Ê¡etO2- _wandbruntime‚2291  _runtime‚2291c.54V‘Ð /loss/FBt2/detection_loss‚0.6548503366644893 .loss/FBt4/detection_loss‚2.720718207458655 %loss/total_loss‚47.28080656133132 epoch‚4 _step‚203 0loss/FRt3/assignment_loss‚3.7632150031053104 0loss/SRqqt4/detection_loss‚2.261779030546164 .loss/FBt3/detection_loss‚2.626268148422241 .loss/FRt2/detection_loss‚1.669491157763534 0loss/FRt2/assignment_loss‚2.3343247489796743 2loss/SRqqt4/assignment_loss‚1.8718954670505645 0loss/FBt4/assignment_loss‚0.5180863178308451 1loss/SRqqt1/detection_loss‚1.7264193553300131 _runtime‚2291.114976294 .loss/FRt4/assignment_loss‚4.36606602565102 % _timestamp‚1.7883915353591926e+09 2loss/SRqqt1/assignment_loss‚1.5261027256647746 1loss/SRqqt2/assignment_loss‚1.487016280502081 0loss/FBt1/assignment_loss‚0.2371622101513172 .loss/FRt4/detection_loss‚2.625091280626214 /loss/FBt1/detection_loss‚2.3847456262225197 0loss/FRt1/assignment_loss‚3.7481724299882586 trainer/global_step‚2499 1loss/FBt2/assignment_loss‚0.22314071191444892 .loss/FRt1/detection_loss‚2.247101274289583 0loss/SRqqt2/detection_loss‚1.199349096596241 2loss/SRqqt3/assignment_loss‚1.8461253866553307 0loss/FBt3/assignment_loss‚0.4720923008291506 /loss/FRt3/detection_loss‚2.6456961922156506 1loss/SRqqt3/detection_loss‚2.1258972468732416æq0½[’Õ .loss/FRt2/detection_loss‚1.669491157763534 0loss/FRt2/assignment_loss‚2.3343247489796743 2loss/SRqqt4/assignment_loss‚1.8718954670505645 0loss/FBt4/assignment_loss‚0.5180863178308451 1loss/SRqqt1/detection_loss‚1.7264193553300131 _runtime‚2291.114976294 .loss/FRt4/assignment_loss‚4.36606602565102 % _timestamp‚1.7883915353591926e+09 2loss/SRqqt1/assignment_loss‚1.5261027256647746 1loss/SRqqt2/assignment_loss‚1.487016280502081 0loss/FBt1/assignment_loss‚0.2371622101513172 .loss/FRt4/detection_loss‚2.625091280626214 /loss/FBt1/detection_loss‚2.3847456262225197 0loss/FRt1/assignment_loss‚3.7481724299882586 trainer/global_step‚2499 1loss/FBt2/assignment_loss‚0.22314071191444892 .loss/FRt1/detection_loss‚2.247101274289583 0loss/SRqqt2/detection_loss‚1.199349096596241 2loss/SRqqt3/assignment_loss‚1.8461253866553307 0loss/FBt3/assignment_loss‚0.4720923008291506 /loss/FRt3/detection_loss‚2.6456961922156506 1loss/SRqqt3/detection_loss‚2.1258972468732416 /loss/FBt2/detection_loss‚0.6548503366644893 .loss/FBt4/detection_loss‚2.720718207458655 %loss/total_loss‚47.28080656133132 epoch‚4 _step‚203 0loss/FRt3/assignment_loss‚3.7632150031053104 0loss/SRqqt4/detection_loss‚2.261779030546164 .loss/FBt3/detection_loss‚2.626268148422241Ë!á\Ø“‚(:’ óàâÔàѼu gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.479' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.495800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0µr:D”‚(:¹ óàâÔ´öÆw disk.sda2.out‚0 disk.dm-0.in‚ 80.87890625 disk.sda2.in‚0) proc.memory.percent‚17.64359474182129 network.recv‚804135) disk./.usagePercent‚70.08095091356596 disk.sda3.out‚ 93.80859375 disk.rbd1.out‚ 1.2890625 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 80.87890625 disk.sda3.in‚0.109375! disk.rbd2.out‚214.06591796875 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.rbd1.in‚0% disk.nvme0n1.out‚12628.2060546875 disk.sda.out‚ 93.80859375' disk.nvme0c0n1.out‚12628.2060546875$ memory_percent‚89.43591117858887" proc.memory.rssMB‚ 2890.7265625 cpu‚25.23870063345217 disk.rbd3.out‚ 1.44140625" disk.dm-0.out‚12628.2060546875 disk.rbd2.in‚0 disk.nvme0n1.in‚ 80.87890625 disk.rbd0.in‚0( proc.memory.availableMB‚ 1730.8203125 disk.rbd3.in‚0 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.dm-1.out‚ 93.80859375 proc.cpu.threads‚17 disk.dm-1.in‚0.109375 network.sent‚ 4.906219e+06% disk./.usageGB‚1044.2568130493164¥ãk~•‚(:’ ‚áâÔ¦Êu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.191' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.438200000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0S,=:–‚(:¯ ‚áâÔøËÓx disk.rbd3.in‚0 disk.sda.out‚94.3125 network.recv‚809626 disk.drbd1777.out‚0# disk.dm-0.out‚12727.09912109375 disk.sda3.out‚94.3125 disk.dm-1.in‚0.109375) proc.memory.availableMB‚ 1730.79296875" proc.memory.rssMB‚ 2890.7265625! disk.nvme0c0n1.in‚ 80.87890625! disk.rbd2.out‚219.50341796875 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0.109375 disk.nvme0n1.in‚ 80.87890625& disk.nvme0n1.out‚12727.09912109375# disk./.usageGB‚1044.25687789917 disk.rbd3.out‚ 1.44140625 disk.dm-0.in‚ 80.87890625 disk.rbd1.in‚0 disk.sda.in‚0.109375 network.sent‚ 4.992594e+06) disk./.usagePercent‚70.08095526569404 disk.drbd1777.in‚0 disk.dm-1.out‚94.3125 disk.rbd1.out‚ 1.2890625 disk.rbd0.out‚0.078125$ memory_percent‚89.43607807159424) proc.memory.percent‚17.64359474182129 disk.sda1.in‚0 cpu‚25.24123221759208 proc.cpu.threads‚17( disk.nvme0c0n1.out‚12727.09912109375 disk.rbd2.in‚0 disk.sda1.out‚0zÖu—‚(:’ ‘áâÔíè§u gpu.0.gpu‚35.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚148.496' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.699200000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢r±C˜‚(:¸ ‘áâÔÁõðw) disk./.usagePercent‚70.08077759646528& disk.nvme0c0n1.out‚12825.724609375 disk.rbd3.in‚0 disk.dm-0.in‚ 80.87890625$ memory_percent‚89.43455219268799 disk.sda2.out‚0 disk.dm-1.out‚ 94.93359375$ disk.nvme0n1.out‚12825.724609375 disk.rbd3.out‚ 1.44140625 disk.rbd1.in‚0) proc.memory.percent‚17.64359474182129 disk.rbd0.out‚0.078125! disk.dm-0.out‚12825.724609375 disk.sda3.in‚0.109375 proc.cpu.threads‚17 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.drbd1777.in‚0) proc.memory.availableMB‚ 1731.04296875 network.sent‚ 4.997957e+06 network.recv‚813420% disk./.usageGB‚1044.2542304992676! disk.rbd2.out‚223.82763671875 disk.nvme0n1.in‚ 80.87890625 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.rbd2.in‚0 disk.sda.out‚ 94.93359375! disk.nvme0c0n1.in‚ 80.87890625 disk.sda2.in‚0 disk.sda3.out‚ 94.93359375 disk.dm-1.in‚0.109375 cpu‚25.243669767157847 disk.rbd1.out‚ 1.2890625" proc.memory.rssMB‚ 2890.7265625ªÕXz2™- _wandbruntime‚2332  _runtime‚2332£UuÁÇšÁ & lr-AdamW/pg1‚0.0012305314971723005 & lr-AdamW/pg2‚0.0012305314971723005 trainer/global_step‚2549 % _timestamp‚1.7883915765507057e+09 _runtime‚2332.306663974 _step‚204"y9Ì›Æ & lr-AdamW/pg1‚0.0012305314971723005 & lr-AdamW/pg2‚0.0012305314971723005 trainer/global_step‚2549 % _timestamp‚1.7883915765507057e+09 _runtime‚2332.306663974 _step‚204Ì6Ló¥2œ- _wandbruntime‚2332  _runtime‚2332ëÒP ÞO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚54371454 _runtime‚2332.310986883 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚173703639754752 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2204621995520 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚205 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚173703639754752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚171602236300288 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚98086426 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 223251456 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚169528615279324 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚98109026 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚43714972 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚171602219260928 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26786228 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚175185980750848 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2089701515360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚54394052 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2101616672768 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚532 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚43714972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚26786231 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7883915765517833e+09 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 248155136 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚172981333851648 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚169528406243424 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 trainer/global_step‚2549 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚171602236300288 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚173703852973056 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚171602219260928 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2101420493824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚40458794 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚167438704728064 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚54394052 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚43714974 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚43714974 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚172981358755328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚175185732595712 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚167438721767424 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13672031 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243BBáé'žãO VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 248155136 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚172981333851648 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚169528406243424 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 trainer/global_step‚2549 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚171602236300288 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚173703852973056 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚171602219260928 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2101420493824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚40458794 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚167438704728064 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚54394052 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚43714974 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚43714974 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚172981358755328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚175185732595712 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚167438721767424 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13672031 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚54371454 _runtime‚2332.310986883 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚173703639754752 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2204621995520 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚205 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚173703639754752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚171602236300288 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚98086426 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 223251456 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚169528615279324 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚98109026 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚43714972 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚171602219260928 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26786228 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚175185980750848 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2089701515360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚54394052 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2101616672768 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚532 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚43714972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚26786231 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7883915765517833e+09 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540Í’0 ý2Ÿ- _wandbruntime‚2333  _runtime‚2333;kÜî& èM J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚43731492 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2090495398308 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚169593008547056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚171667258297856 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2102415076864 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2090687395056 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚54414950 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚173769690414080 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚54414950 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚173046859347968 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚175252108251136 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2102415076864 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚171667258297856 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚43731492 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚40473283 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚173046884251648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231626752 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚173769690414080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚98146444 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚54392326 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚169592799510948 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2205480529920 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚167502321152000 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚2549 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚98123818 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚54392326 % _timestamp‚1.7883915773007615e+09 _runtime‚2333.057293701 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256530432 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2205248903168 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13677536 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2102218884608 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚26796265 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚98123818 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13677021 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚43731494 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚171667275337216 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2102218884608 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚175252364781568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚98146444 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚43731494 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507Y—z@¡íM ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚173769690414080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚98146444 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚54392326 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚169592799510948 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2205480529920 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚167502321152000 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚2549 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚98123818 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚54392326 % _timestamp‚1.7883915773007615e+09 _runtime‚2333.057293701 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256530432 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2205248903168 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13677536 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2102218884608 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚26796265 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚98123818 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freeR÷¢u³d‚13677021 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚43731494 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚171667275337216 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2102218884608 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚175252364781568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚98146444 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚43731494 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚43731492 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2090495398308 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚169593008547056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚171667258297856 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2102415076864 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2090687395056 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚54414950 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚173769690414080 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚54414950 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚173046859347968 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚175252108251136 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2102415076864 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚171667258297856 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚43731492 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚40473283 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚173046884251648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231626752 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3ÎÖ¦÷ñ2¢- _wandbruntime‚2333  _runtime‚2333窙AZ£Ô 2loss/SRqqt2/assignment_loss‚1.2737075131519564 0loss/FRt2/assignment_loss‚2.0094521468461948 2loss/SRqqt4/assignment_loss‚1.9876212363059704 /loss/FRt2/detection_loss‚1.8637261263333087 /loss/FRt4/detection_loss‚2.5597482548243757 trainer/global_step‚2549 epoch‚4 /loss/SRqqt1/assignment_loss‚1.6403749396153 /loss/FRt3/detection_loss‚2.4945892396937595 1loss/SRqqt1/detection_loss‚1.8056087093342335 .loss/FBt4/detection_loss‚2.519294662134988 $loss/total_loss‚45.7963111686152 _step‚207 0loss/FRt3/assignment_loss‚3.5263645012726945 2loss/FBt4/assignment_loss‚0.057352362383556156 /loss/FBt1/detection_loss‚2.7263510208576918 % _timestamp‚1.7883915773025131e+09 _runtime‚2333.058344101 /loss/FRt4/assignment_loss‚4.225469058838444 1loss/FBt1/assignment_loss‚0.17372632477781735 0loss/FBt2/assignment_loss‚0.2238160275477584 1loss/FBt3/assignment_loss‚0.16655524120557988 1loss/SRqqt2/detection_loss‚0.9151640589482017 0loss/SRqqt3/detection_loss‚1.905487783504336 .loss/FBt3/detection_loss‚2.644954741001129 1loss/SRqqt4/detection_loss‚2.4651996484169594 0loss/FRt1/assignment_loss‚3.7291140686506514 2loss/SRqqt3/assignment_loss‚1.7965776180283408 /loss/FBt2/detection_loss‚0.5317965881355727 .loss/FRt1/detection_loss‚2.554259296806379Qƒí2_¤Ù % _timestamp‚1.7883915773025131e+09 _runtime‚2333.058344101 /loss/FRt4/assignment_loss‚4.225469058838444 1loss/FBt1/assignment_loss‚0.17372632477781735 0loss/FBt2/assignment_loss‚0.2238160275477584 1loss/FBt3/assignment_loss‚0.16655524120557988 1loss/SRqqt2/detection_loss‚0.9151640589482017 0loss/SRqqt3/detection_loss‚1.905487783504336 .loss/FBt3/detection_loss‚2.644954741001129 1loss/SRqqt4/detection_loss‚2.4651996484169594 0loss/FRt1/assignment_loss‚3.7291140686506514 2loss/SRqqt3/assignment_loss‚1.7965776180283408 /loss/FBt2/detection_loss‚0.5317965881355727 .loss/FRt1/detection_loss‚2.554259296806379 2loss/SRqqt2/assignment_loss‚1.2737075131519564 0loss/FRt2/assignment_loss‚2.0094521468461948 2loss/SRqqt4/assignment_loss‚1.9876212363059704 /loss/FRt2/detection_loss‚1.8637261263333087 /loss/FRt4/detection_loss‚2.5597482548243757 trainer/global_step‚2549 epoch‚4 /loss/SRqqt1/assignment_loss‚1.6403749396153 /loss/FRt3/detection_loss‚2.4945892396937595 1loss/SRqqt1/detection_loss‚1.8056087093342335 .loss/FBt4/detection_loss‚2.519294662134988 $loss/total_loss‚45.7963111686152 _step‚207 0loss/FRt3/assignment_loss‚3.5263645012726945 2loss/FBt4/assignment_loss‚0.057352362383556156 /loss/FBt1/detection_loss‚2.7263510208576918ÏC¦ƒ œ¥‚(:‘  áâÔβ¦u gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚96.549' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚19.309800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XùoFC¦‚(:¸  áâÔíåÛw disk.drbd1777.in‚0) proc.memory.availableMB‚ 1730.71484375) proc.memory.percent‚17.64359474182129% disk./.usageGB‚1044.2542686462402 disk.sda3.out‚ 95.3984375 disk.nvme0n1.in‚ 80.87890625 disk.rbd3.in‚0 network.sent‚ 5.004104e+06 disk.dm-0.in‚ 80.87890625 disk.sda3.in‚0.109375 disk.rbd1.out‚ 1.2890625 proc.cpu.threads‚17) disk./.usagePercent‚70.08078015654063 disk.sda1.in‚0 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚12826.9716796875 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚89.43655490875244 cpu‚25.246268996143833" disk.dm-0.out‚12826.9716796875 disk.dm-1.out‚ 95.3984375! disk.rbd2.out‚229.26123046875 disk.rbd0.in‚0 disk.rbd3.out‚ 1.44140625" proc.memory.rssMB‚ 2890.7265625 network.recv‚818180 disk.rbd2.in‚0% disk.nvme0n1.out‚12826.9716796875 disk.sda1.out‚0 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 80.87890625 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.sda.in‚0.109375 disk.sda.out‚ 95.3984375ío`E§‚(:’ ¯áâÔ¬ÅÓu gpu.0.gpu‚40.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.383' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.076600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒÿ‹0¨‚(:¥ ¯áâÔ±¶±y$ memory_percent‚89.43707942962646" proc.memory.rssMB‚ 2890.7265625' disk.nvme0c0n1.out‚12925.3935546875 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 80.87890625 cpu‚25.248695823225464 network.sent‚ 5.09199e+06 network.recv‚824459) disk./.usagePercent‚70.08077913251049% disk./.usageGB‚1044.2542533874512 disk.rbd0.in‚0) proc.memory.availableMB‚ 1730.62890625 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.sda.in‚0.109375 disk.drbd1777.out‚0 disk.sda3.out‚96.25 disk.rbd2.in‚0) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17 disk.sda1.out‚0 disk.dm-0.in‚ 80.87890625 disk.rbd1.out‚ 1.2890625% disk.nvme0n1.out‚12925.3935546875 disk.sda.out‚96.25 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 disk.dm-1.out‚96.25 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0" disk.dm-0.out‚12925.3935546875 disk.rbd2.out‚ 233.60546875 disk.nvme0n1.in‚ 80.87890625IŒ°œ©‚(:‘ ¾áâÔ“³u gpu.0.gpu‚30.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚89.301' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.860200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿ!¨Aª‚(:¶ ¾áâÔ˜íx cpu‚25.251155103058146$ disk./.usageGB‚1044.254379272461 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.dm-0.in‚ 80.87890625 disk.rbd1.out‚ 1.2890625" proc.memory.rssMB‚ 2890.7265625 proc.cpu.threads‚17 disk.sda3.in‚0.109375 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 80.87890625 disk.sda2.in‚0 network.recv‚828265 disk.rbd2.out‚ 238.9140625 disk.rbd3.in‚0 disk.dm-1.in‚0.109375 disk.sda1.in‚0$ memory_percent‚89.43560123443604 disk.nvme0n1.in‚ 80.87890625% disk.nvme0n1.out‚12926.2255859375 disk.sda2.out‚0 disk.sda3.out‚ 97.04296875 disk.dm-1.out‚ 97.04296875 disk.sda.out‚ 97.04296875 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0) proc.memory.availableMB‚ 1730.87109375 network.sent‚ 5.097381e+06) disk./.usagePercent‚70.08078758075911 disk.rbd3.out‚ 1.44140625" disk.dm-0.out‚12926.2255859375 disk.sda1.out‚0' disk.nvme0c0n1.out‚12926.2255859375) proc.memory.percent‚17.64359474182129TýÖ2«- _wandbruntime‚2374  _runtime‚2374„·.ǬÁ % _timestamp‚1.7883916183800075e+09 _runtime‚2374.135976967 _step‚208 & lr-AdamW/pg1‚0.0012079860602891568 & lr-AdamW/pg2‚0.0012079860602891568 trainer/global_step‚2599(Å)QÌ­Æ & lr-AdamW/pg1‚0.0012079860602891568 & lr-AdamW/pg2‚0.0012079860602891568 trainer/global_step‚2599 % _timestamp‚1.7883916183800075e+09 _runtime‚2374.135976967 _step‚208ÐÈV 2®- _wandbruntime‚2374  _runtime‚2374ß%« ä'¯ÞO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2141564040192 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚176257608667648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚172748096791632 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚170618690996224 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚99959378 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚176257633571328 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚170618673956864 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚174854171107328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚176995538968576 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2246542120448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚531 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13928960 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚178504150788096 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2246786343424 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2141367861248 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚44540972 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚55418406 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚27287931 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚178504419914752 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚174854188146688 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚2374.137277448 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚174854188146688 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚176995538968576 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚44540974 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚44540972 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2129422834768 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚2599 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2141564040192 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚41217422 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13929491 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 % _timestamp‚1.7883916183810666e+09 ¨èºé'°ãO `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚174854188146688 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚176995538968576 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚44540974 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚44540972 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2129422834768 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚2599 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2141564040192 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚41217422 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13929491 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 % _timestamp‚1.7883916183810666e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2141564040192 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚176257608667648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚172748096791632 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚170618690996224 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚99959378 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚176257633571328 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚170618673956864 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚174854171107328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚176995538968576 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2246542120448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚531 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13928960 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚178504150788096 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2246786343424 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2141367861248 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚44540972 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚55418406 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚27287931 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚178504419914752 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚174854188146688 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚2374.137277448ÑPÿd*2±- _wandbruntime‚2374  _runtime‚2374É~ôe²êM I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonit(C/ú‹&or.on_train_batch_end/inactive_split_bytes.large_pool.freed‚176323134163968 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚174919227183616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 % _timestamp‚1.7883916191260593e+09 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚27297965 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚170682290380800 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚44557492 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚178570784507904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2142166252032 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 100019396 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2142166252032 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚44557492 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚178570521686016 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13934059 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 100019396 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚520 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚176323159067648 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚44557494 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚2374.882630903 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚99996770 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚174919227183616 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41232541 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2142362444288 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚44557494 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚172812699095264 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚517 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚170682273341440 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚172812490059156 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41232021 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚55439278 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256+Oõ&³ïM \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2130408714464 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚55461902 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚55439278 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 100019396 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚520 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚176323159067648 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚44557494 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚2374.882630903 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚99996770 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚174919227183616 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41232541 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2142362444288 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚44557494 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚172812699095264 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚517 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚170682273341440 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚172812490059156 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41232021 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚55439278 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚176323134163968 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚174919227183616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 % _timestamp‚1.7883916191260593e+09 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚27297965 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚170682290380800 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚44557492 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚178570784507904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2142166252032 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 100019396 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2142166252032 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚44557492 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚178570521686016 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13934059 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FÃË¿Å & lr-AdamW/pg1‚0.0011851494910467462 & lr-AdamW/pg2‚0.0011851494910467462 trainer/global_step‚2649 % _timestamp‚1.7883916604397216e+09 _runtime‚ 2416.19567095 _step‚212Ô—TÐ<2À- _wandbruntime‚2416  _runtime‚2416ë(xµ¯ÁâO YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚175967996375740 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚56465358 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚175967787339840 % _timestamp‚1.7883916604408417e+09 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚173798660225024 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚180287651400704 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2169336150716 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚178106122953728 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 _runtime‚2416.197046743 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚27789631 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚181822635272192 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 101854930 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚56487956 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚45366972 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14187354 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 101832330 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚180287651400704 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚56487956 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚45366974 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚180287438182400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2181511407616 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚180287438182400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2181315228672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2181315228672 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚2649 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚178106139993088 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚173798643185664 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚179533883483648 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚27789628 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2288993914368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚45366972 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚178106139993088 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚181822902301696ü¼í'ÂçO MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚180287438182400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2181511407616 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚180287438182400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2181315228672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2181315228672 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚2649 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚178106139993088 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚173798643185664 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚179533883483648 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚27789628 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2288993914368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚45366972 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚178106139993088 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚181822902301696 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚175967996375740 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚56465358 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚175967787339840 % _timestamp‚1.7883916604408417e+09 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚173798660225024 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚180287651400704 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2169336150716 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚178106122953728 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 _runtime‚2416.197046743 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚27789631 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚181822635272192 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 101854930 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚56487956 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚45366972 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14187354 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 101832330 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚180287651400704 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚56487956 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚45366974Õã#Iv2Ã- _wandbruntime‚2416  _runtime‚2416 …Áò&ÄìM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2182309811712 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 101892348 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚178171179030016 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚180353275610112 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2182309811712 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚181889000568832 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41992184 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 101892348 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2170130033872 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚56508854 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚181889267585024 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2182113619456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚56486230 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41992711 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚56486230 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚2649 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚45383494 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚45383492 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚56508854 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚179599408979968 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚176032389643472 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚45383492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2289591588864 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚45383494 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚173862242570240 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 % _timestamp‚1.7883916611982732e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚178171179030016 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14192522 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚173862259609600 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2169938037124 _runtime‚2416.954891564 _step‚214 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚27799662 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14192522 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚173862259609600 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2169938037124 _runtime‚2416.954891564 _step‚214 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚27799662 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 101892348 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚178171179030016 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚180353275610112 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2182309811712 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚181889000568832 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41992184 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 101892348 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2170130033872 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚56508854 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚181889267585024 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2182113619456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚56486230 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41992711 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚56486230 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 t»Õ?›d rainer/global_step‚2649 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚45383494 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚45383492 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚56508854 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚179599408979968 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚176032389643472 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚45383492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2289591588864 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚45383494 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚173862242570240 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 % _timestamp‚1.7883916611982732e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚178171179030016Öºâ©2Æ- _wandbruntime‚2416  _runtime‚2416´EtWÇÑ 0loss/FRt4/assignment_loss‚4.1800971203728725 0loss/FBt2/assignment_loss‚0.1466450238475668 0loss/SRqqt4/detection_loss‚2.344408515133435 epoch‚4 _runtime‚2416.955907388 %loss/total_loss‚46.25020488552936 0loss/FRt3/assignment_loss‚3.4821168261689026 2loss/SRqqt2/assignment_loss‚1.3823260043966858 0loss/FBt1/assignment_loss‚0.1787437966385092 /loss/FRt3/detection_loss‚2.5471837052277158 0loss/SRqqt2/detection_loss‚1.052104755085619 # _timestamp‚1.78839166120008e+09 2loss/SRqqt1/assignment_loss‚1.8207050344720483 1loss/SRqqt1/detection_loss‚1.7743668026477097 /loss/FBt1/detection_loss‚2.2849645277728206 1loss/SRqqt4/assignment_loss‚2.176088928987708 /loss/FRt1/detection_loss‚2.4365310381694014 /loss/FRt2/detection_loss‚1.8740207921345156 .loss/FRt4/detection_loss‚2.383558258022133 1loss/SRqqt3/detection_loss‚2.0078747966326773 .loss/FBt3/detection_loss‚2.769476572672526 trainer/global_step‚2649 _step‚215 0loss/FRt2/assignment_loss‚2.0129643279117544 2loss/SRqqt3/assignment_loss‚1.6476563810138032 1loss/FBt3/assignment_loss‚0.32798536154182834 0loss/FRt1/assignment_loss‚3.5097816349512123 1loss/FBt4/assignment_loss‚0.23097667500705404 -loss/FBt2/detection_loss‚0.68008553898646 .loss/FBt4/detection_loss‚2.999542467734393ÂîÙ\ÈÖ 0loss/FRt3/assignment_loss‚3.4821168261689026 2loss/SRqqt2/assignment_loss‚1.3823260043966858 0loss/FBt1/assignment_loss‚0.1787437966385092 /loss/FRt3/detection_loss‚2.5471837052277158 0loss/SRqqt2/detection_loss‚1.052104755085619 # _timestamp‚1.78839166120008e+09 2loss/SRqqt1/assignment_loss‚1.8207050344720483 1loss/SRqqt1/detection_loss‚1.7743668026477097 /loss/FBt1/detection_loss‚2.2849645277728206 1loss/SRqqt4/assignment_loss‚2.176088928987708 /loss/FRt1/detection_loss‚2.4365310381694014 /loss/FRt2/detection_loss‚1.8740207921345156 .loss/FRt4/detection_loss‚2.383558258022133 1loss/SRqqt3/detection_loss‚2.0078747966326773 .loss/FBt3/detection_loss‚2.769476572672526 trainer/global_step‚2649 _step‚215 0loss/FRt2/assignment_loss‚2.0129643279117544 2loss/SRqqt3/assignment_loss‚1.6476563810138032 1loss/FBt3/assignment_loss‚0.32798536154182834 0loss/FRt1/assignment_loss‚3.5097816349512123 1loss/FBt4/assignment_loss‚0.23097667500705404 -loss/FBt2/detection_loss‚0.68008553898646 .loss/FBt4/detection_loss‚2.999542467734393 0loss/FRt4/assignment_loss‚4.1800971203728725 0loss/FBt2/assignment_loss‚0.1466450238475668 0loss/SRqqt4/detection_loss‚2.344408515133435 epoch‚4 _runtime‚2416.955907388 %loss/total_loss‚46.25020488552936×è)ød‘É‚(:† úáâÔŸôÑu gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚156.834' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.3668 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚02µ>s6Ê‚(:« úáâÔòì—x disk.rbd1.in‚0 disk.rbd1.out‚ 1.2890625 disk.nvme0n1.in‚ 80.8828125& disk.nvme0n1.out‚13225.58349609375 disk.rbd0.in‚0 disk.rbd3.out‚ 1.44140625 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.sda.out‚ 99.7109375 disk.nvme0c0n1.in‚ 80.8828125" proc.memory.rssMB‚ 2890.7265625 disk.dm-0.in‚ 80.8828125# disk.dm-0.out‚13225.58349609375 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd0.out‚0.078125 proc.cpu.threads‚17) disk./.usagePercent‚70.08099776294472 disk.sda3.out‚ 99.7109375 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚13225.58349609375 network.sent‚ 5.201988e+06! disk.rbd2.out‚241.14111328125 disk.rbd3.in‚0 disk.sda2.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0) proc.memory.percent‚17.64359474182129 cpu‚25.26089226071003 network.recv‚846675$ disk./.usageGB‚1044.257511138916 disk.sda3.in‚0.109375 disk.sda.in‚0.109375" memory_percent‚89.434814453125 proc.memory.availableMB‚1731 disk.dm-1.out‚ 99.7109375äµü`‘Ë‚(:† ‰ââÔÓú½u gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.145' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.029 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚01YøIÌ‚(:¾ ‰ââÔõÝŠy disk.rbd1.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚241.23095703125 disk.sda3.out‚ 100.36328125 disk.rbd0.in‚0 disk.nvme0c0n1.in‚ 80.8828125 disk.dm-1.out‚ 100.36328125$ memory_percent‚89.43636417388916" proc.memory.rssMB‚ 2890.7265625 disk.sda.out‚ 100.36328125) disk./.usagePercent‚70.08099648290705 disk.rbd1.out‚ 1.2890625 disk.sda.in‚0.109375 disk.rbd3.in‚0% disk./.usageGB‚1044.2574920654297 disk.drbd1777.out‚0 cpu‚25.263359512264564 disk.nvme0n1.in‚ 80.8828125 disk.rbd3.out‚ 1.44140625) proc.memory.availableMB‚ 1730.74609375& disk.nvme0n1.out‚13323.93115234375 disk.sda1.in‚0 disk.sda2.in‚0 disk.dm-0.in‚ 80.8828125 disk.sda3.in‚0.109375 proc.cpu.threads‚17 disk.sda1.out‚0 network.sent‚ 5.288503e+06 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚13323.93115234375 disk.sda2.out‚0# disk.dm-0.out‚13323.93115234375) proc.memory.percent‚17.64359474182129 network.recv‚851554 disk.rbd0.out‚0.078125 disk.dm-1.in‚0.109375ÊIÊ2Í- _wandbruntime‚2458  _runtime‚2458“‡’.ÇÎÁ & lr-AdamW/pg1‚0.0011620408564069575 & lr-AdamW/pg2‚0.0011620408564069575 trainer/global_step‚2699 % _timestamp‚1.7883917023308928e+09 _runtime‚2458.086930703 _step‚216}c {ÌÏÆ trainer/global_step‚2699 % _timestamp‚1.7883917023308928e+09 _runtime‚2458.086930703 _step‚216 & lr-AdamW/pg1‚0.0011620408564069575 & lr-AdamW/pg2‚0.0011620408564069575Ø5/º,2Ð- _wandbruntime‚2458  _runtime‚2458¥o^‹è'ÑâO P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚185140985307136 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚182810183203328 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚182810158299648 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚176978612414464 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚181358074800128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚28291331 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2221458775040 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚57534908 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚46192974 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14444648 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚57512310 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2221262596096 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚181358091839488 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚28291328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚181358091839488 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2221262596096 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚519 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚516 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚46192972 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2331064939008 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚179187477888048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚183579550614528 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚183579337396224 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚179187686923948 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _step‚217 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚57534908 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚46192974 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14444648 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚57512310 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2221262596096 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚181358091839488 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚28291328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚181358091839488 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2221262596096 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚519 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚516 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚46192972 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2331064939008 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚179187477888048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚183579550614528 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚183579337396224 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚179187686923948 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _step‚217 J6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚181423130876416 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚42751008 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14449643 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚185207346702336 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚57555806 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚182875683795968 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚183645388055552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚46209492 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28301365 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚28301362 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚185207615815680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚179252080191680 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚177042228838400 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚2699 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2222060986880 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚182875708699648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚42750470 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 103742674 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 103765300 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14449108 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2209851353280 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚183645388055552 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚181423113837056 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚179251871155572 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883917030818567e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚46209494 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824;ïXÝ÷&ÕñM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 103765300 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 _runtime‚2458.838371807 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚57533182 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚181423130876416 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚42751008 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14449643 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚185207346702336 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚57555806 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚182875683795968 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚183645388055552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚46209492 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28301365 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚28301362 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚185207615815680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚179252080191680 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚177042228838400 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚2699 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2222060986880 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚182875708699648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚42750470 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 103742674 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 103765300 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14449108 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2209851353280 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚183645388055552 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚181423113837056 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚179251871155572 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883917030818567e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚46209494ÚÝÁˆ¹2Ö- _wandbruntime‚2458  _runtime‚2458çx¶š]×× 2loss/SRqqt3/assignment_loss‚1.6702342331842628 .loss/FRt3/detection_loss‚2.629616452789888 /loss/FBt2/detection_loss‚0.5511418712497662 %loss/total_loss‚46.35401262652997 trainer/global_step‚2699 /loss/FRt1/assignment_loss‚3.519077264449813 0loss/FRt3/assignment_loss‚3.8603828120522383 /loss/FBt3/detection_loss‚2.8526265892115505 0loss/FBt3/assignment_loss‚0.4987503792878918 /loss/FRt2/detection_loss‚1.6246617803459658 1loss/SRqqt3/detection_loss‚1.9149188082232032 /loss/FBt1/detection_loss‚2.9976014494895935 _step‚219 /loss/FRt1/detection_loss‚2.3492638560181316 1loss/SRqqt2/detection_loss‚0.9121557893636434 0loss/SRqqt4/detection_loss‚2.454415417031238 _runtime‚2458.839553243 0loss/FRt4/assignment_loss‚4.3809844721918525 1loss/FBt4/assignment_loss‚0.20976494995120448 /loss/FBt4/detection_loss‚2.5015349785486856 2loss/SRqqt2/assignment_loss‚1.2439023960462732 /loss/FRt4/detection_loss‚2.3941004293552344 % _timestamp‚1.7883917030837455e+09 /loss/FRt2/assignment_loss‚2.238085896945467 1loss/FBt2/assignment_loss‚0.16870437647637088 epoch‚4 2loss/SRqqt1/assignment_loss‚1.6715763751417398 2loss/SRqqt4/assignment_loss‚1.7883603034312265 1loss/FBt1/assignment_loss‚0.04607635563928246 0loss/SRqqt1/detection_loss‚1.876075390105446hyAbØÜ 2loss/SRqqt1/assignment_loss‚1.6715763751417398 2loss/SRqqt4/assignment_loss‚1.7883603034312265 1loss/FBt1/assignment_loss‚0.04607635563928246 0loss/SRqqt1/detection_loss‚1.876075390105446 2loss/SRqqt3/assignment_loss‚1.6702342331842628 .loss/FRt3/detection_loss‚2.629616452789888 /loss/FBt2/detection_loss‚0.5511418712497662 %loss/total_loss‚46.35401262652997 trainer/global_step‚2699 /loss/FRt1/assignment_loss‚3.519077264449813 0loss/FRt3/assignment_loss‚3.8603828120522383 /loss/FBt3/detection_loss‚2.8526265892115505 0loss/FBt3/assignment_loss‚0.4987503792878918 /loss/FRt2/detection_loss‚1.6246617803459658 1loss/SRqqt3/detection_loss‚1.9149188082232032 /loss/FBt1/detection_loss‚2.9976014494895935 _step‚219 /loss/FRt1/detection_loss‚2.3492638560181316 1loss/SRqqt2/detection_loss‚0.9121557893636434 0loss/SRqqt4/detection_loss‚2.454415417031238 _runtime‚2458.839553243 0loss/FRt4/assignment_loss‚4.3809844721918525 1loss/FBt4/assignment_loss‚0.20976494995120448 /loss/FBt4/detection_loss‚2.5015349785486856 2loss/SRqqt2/assignment_loss‚1.2439023960462732 /loss/FRt4/detection_loss‚2.3941004293552344 % _timestamp‚1.7883917030837455e+09 /loss/FRt2/assignment_loss‚2.238085896945467 1loss/FBt2/assignment_loss‚0.16870437647637088 epoch‚4ÛP ¤Ù‚(:’ ˜ââÔ§°¶u gpu.0.gpu‚35.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚101.193' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚20.238599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‚+ †7Ú‚(:¬ ˜ââÔõÕÑx network.recv‚856170 disk.dm-1.out‚ 100.65625$ memory_percent‚89.43636417388916) proc.memory.availableMB‚ 1730.74609375" proc.memory.rssMB‚ 2890.7265625 disk.sda3.out‚ 100.65625 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.dm-0.out‚13324.63671875 disk.rbd1.in‚0 disk.rbd1.out‚ 1.2890625 disk.rbd0.in‚0 disk.dm-0.in‚ 80.8828125 disk.sda3.in‚0.109375! disk.rbd2.out‚241.23486328125 disk.sda.in‚0.109375 disk.nvme0n1.in‚ 80.8828125 disk.sda1.in‚0 disk.nvme0c0n1.in‚ 80.8828125 proc.cpu.threads‚17) disk./.usagePercent‚70.08104358829335% disk.nvme0c0n1.out‚13324.63671875 disk.sda2.in‚0% disk./.usageGB‚1044.2581939697266 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.rbd3.in‚0 network.sent‚ 5.294658e+06# disk.nvme0n1.out‚13324.63671875 disk.sda.out‚ 100.65625 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625) proc.memory.percent‚17.64359474182129 cpu‚25.265630580699092|åÛ‚(:’ §ââÔ¶¿Ìu gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.553' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.310599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0·Æ¥EÜ‚(:º §ââÔ˜Ê×x disk.drbd1777.out‚0( disk.nvme0c0n1.out‚13423.73095703125 cpu‚25.267767551256732 network.sent‚ 5.382396e+06 network.recv‚862428 disk.rbd1.out‚ 1.2890625 disk.rbd2.in‚0! disk.rbd2.out‚241.38037109375 disk.dm-0.in‚ 80.8828125 disk.rbd0.in‚0 disk.rbd3.in‚0" proc.memory.rssMB‚ 2890.7265625) disk./.usagePercent‚70.08133978901047 disk.sda3.in‚0.109375 disk.dm-1.out‚ 101.26953125 disk.rbd1.in‚0& disk.nvme0n1.out‚13423.73095703125 disk.nvme0c0n1.in‚ 80.8828125 disk.sda3.out‚ 101.26953125 disk.sda1.in‚0$ memory_percent‚89.43653106689453 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125& proc.memory.availableMB‚ 1730.71875 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 disk.dm-1.in‚0.109375 disk.sda1.out‚0# disk.dm-0.out‚13423.73095703125) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17$ disk./.usageGB‚1044.262607574463 disk.nvme0n1.in‚ 80.8828125 disk.sda.out‚ 101.26953125ÜImpÝ‚(:’ ¶ââÔ¢ûÕu gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.616' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.723200000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0óï7Þ‚(:¬ ¶ââÔʬx disk.dm-0.in‚ 80.88671875 disk.sda3.in‚0.109375 network.sent‚ 5.387656e+06% disk./.usageGB‚1044.2597846984863 disk.rbd3.in‚0 disk.sda.in‚0.109375 disk.drbd1777.in‚0' disk.nvme0c0n1.out‚13523.5732421875 disk.dm-1.in‚0.109375 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 network.recv‚866050 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.sda3.out‚101.875 disk.rbd1.in‚0! disk.rbd2.out‚241.38037109375( proc.memory.availableMB‚ 1730.4609375) proc.memory.percent‚17.64359474182129 disk.sda.out‚101.875 disk.sda1.out‚0 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.out‚ 1.44140625) disk./.usagePercent‚70.08115034343513" disk.dm-0.out‚13523.5732421875 disk.rbd2.in‚0 cpu‚25.26998421630642 disk.rbd1.out‚ 1.2890625" proc.memory.rssMB‚ 2890.7265625 disk.dm-1.out‚101.875 disk.nvme0n1.in‚ 80.88671875% disk.nvme0n1.out‚13523.5732421875# memory_percent‚89.4381046295166 proc.cpu.threads‚17Ç ‰â2ß- _wandbruntime‚2500  _runtime‚2500™ Ø1ÇàÁ & lr-AdamW/pg2‚0.0011386794504875419 trainer/global_step‚2749 % _timestamp‚1.7883917447201264e+09 _runtime‚2500.476092232 _step‚220 & lr-AdamW/pg1‚0.0011386794504875419À(íÜÌáÆ & lr-AdamW/pg2‚0.0011386794504875419 trainer/global_step‚2749 % _timestamp‚1.7883917447201264e+09 _runtime‚2500.476092232 _step‚220 & lr-AdamW/pg1‚0.0011386794504875419܃6tî2â- _wandbruntime‚2500  _runtime‚2500v¨lÝãâO KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚186871449828352 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 105600834 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2373174429184 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2248778789532 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2261406142464 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2261209963520 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚184610043685888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚58559262 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚186871236610048 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚184610026646528 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚47018974 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚188459363321856 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚182407377472156 trainer/global_step‚2749 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚186871236610048 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚43494735 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚2500.477446995 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚47018974 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47018972 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚519 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚184610026646528 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2261209963520 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚182407168436256 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2373174429184 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2248778789532 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2261406142464 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2261209963520 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚184610043685888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚58559262 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚186871236610048 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚184610026646528 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚47018974 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚188459363321856 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚182407377472156 trainer/global_step‚2749 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚186871236610048 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚43494735 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚2500.477446995 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚47018974 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47018972 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚519 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚184610026646528 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2261209963520 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚182407168436256 JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚186151983515648 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚47035492 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚2501.230278687 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚184675065683456 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚188525737216000 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚186151958611968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚180222181027840 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14706260 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 105638252 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚43509832 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 105638252 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚188525991649280 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚182471770739888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚58580134 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2262008354304 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚43509322 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28803065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚507 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚184675065683456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚58580134°`ƒ£÷&çñM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚2501.230278687 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚184675065683456 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚188525737216000 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚186151958611968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚180222181027840 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14706260 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 105638252 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚43509832 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 105638252 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚188525991649280 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚182471770739888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚58580134 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2262008354304 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚43509322 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28803065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚507 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚184675065683456 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚58580134 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚186151983515648 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚47035492 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713Þ*2˜2è- _wandbruntime‚2501  _runtime‚2501sªðÓŠé× 2loss/SRqqt1/assignment_loss‚1.8799700787694185 2loss/SRqqt3/assignment_loss‚1.7513798264805804 /loss/FRt3/detection_loss‚2.6267336864530306 1loss/SRqqt4/detection_loss‚2.3411626311731926 epoch‚4 $ _timestamp‚1.788391745475699e+09 0loss/FRt2/assignment_loss‚2.2251388213286796 0loss/FBt1/assignment_loss‚0.1606437005839325 .loss/FBt1/detection_loss‚2.ù¢Ó724143259227276 trainer/global_step‚2749 2loss/SRqqt4/assignment_loss‚1.9072458802917858 0loss/FRt4/assignment_loss‚3.9768937266234197 1loss/FBt4/assignment_loss‚0.08892357488886253 .loss/FRt1/detection_loss‚2.337465747239742 /loss/FBt2/detection_loss‚0.5537571256808272 .loss/FBt3/detection_loss‚2.367426625706933 /loss/FBt4/detection_loss‚2.7670657208987643 /loss/FRt2/detection_loss‚1.6887442368703585 /loss/FRt4/detection_loss‚2.6674876484003933 0loss/FBt3/assignment_loss‚0.4262440978253091 _runtime‚2501.231497233 0loss/FRt3/assignment_loss‚3.5297478474216697 2loss/SRqqt2/assignment_loss‚1.3058223565911047 1loss/FBt2/assignment_loss‚0.16034757153078683 1loss/SRqqt2/detection_loss‚0.9287571198203497 %loss/total_loss‚45.87467696949584 _step‚223 0loss/FRt1/assignment_loss‚3.7019196862870074 1loss/SRqqt1/detection_loss‚1.8926732961050015 1loss/SRqqt3/detection_loss‚1.8649827032974087úg§öbêÜ 2loss/SRqqt4/assignment_loss‚1.9072458802917858 0loss/FRt4/assignment_loss‚3.9768937266234197 1loss/FBt4/assignment_loss‚0.08892357488886253 .loss/FRt1/detection_loss‚2.337465747239742 /loss/FBt2/detection_loss‚0.5537571256808272 .loss/FBt3/detection_loss‚2.367426625706933 /loss/FBt4/detection_loss‚2.7670657208987643 /loss/FRt2/detection_loss‚1.6887442368703585 /loss/FRt4/detection_loss‚2.6674876484003933 0loss/FBt3/assignment_loss‚0.4262440978253091 _runtime‚2501.231497233 0loss/FRt3/assignment_loss‚3.5297478474216697 2loss/SRqqt2/assignment_loss‚1.3058223565911047 1loss/FBt2/assignment_loss‚0.16034757153078683 1loss/SRqqt2/detection_loss‚0.9287571198203497 %loss/total_loss‚45.87467696949584 _step‚223 0loss/FRt1/assignment_loss‚3.7019196862870074 1loss/SRqqt1/detection_loss‚1.8926732961050015 1loss/SRqqt3/detection_loss‚1.8649827032974087 2loss/SRqqt1/assignment_loss‚1.8799700787694185 2loss/SRqqt3/assignment_loss‚1.7513798264805804 /loss/FRt3/detection_loss‚2.6267336864530306 1loss/SRqqt4/detection_loss‚2.3411626311731926 epoch‚4 $ _timestamp‚1.788391745475699e+09 0loss/FRt2/assignment_loss‚2.2251388213286796 0loss/FBt1/assignment_loss‚0.1606437005839325 .loss/FBt1/detection_loss‚2.724143259227276 trainer/global_step‚2749ß›2§{ë‚(:’ ÅââÔã³Ùu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.443' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.488600000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¸…¼¥Hì‚(:½ ÅââÔ¨ÓÙx disk.drbd1777.out‚0 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.dm-1.out‚ 102.44140625! disk.nvme0c0n1.in‚ 80.88671875' disk.nvme0c0n1.out‚13524.0517578125 disk.rbd1.in‚0 disk.rbd1.out‚ 1.2890625 disk.drbd1777.in‚0 network.recv‚869534 disk.sda.out‚ 102.44140625 disk.sda2.out‚0 proc.cpu.threads‚17 disk.rbd3.out‚ 1.44140625! disk.rbd2.out‚241.44677734375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125% disk./.usageGB‚1044.2599067687988 disk.rbd3.in‚0 disk.dm-0.in‚ 80.88671875" disk.dm-0.out‚13524.0517578125$ memory_percent‚89.43846225738525" proc.memory.rssMB‚ 2890.7265625 disk.sda3.out‚ 102.44140625 cpu‚25.27225761300511 disk.dm-1.in‚0.109375) disk./.usagePercent‚70.08115853567622 disk.nvme0n1.in‚ 80.88671875 disk.sda1.in‚0 disk.sda2.in‚0 network.sent‚ 5.393082e+06% disk.nvme0n1.out‚13524.0517578125 disk.sda3.in‚0.109375 disk.rbd2.in‚0) proc.memory.availableMB‚ 1730.40234375) proc.memory.percent‚17.64359474182129&²mÅí‚(:’ ÔââÔ÷Þ„v gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.047' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.209400000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ö=QHî‚(:½ ÔââÔó–Èw disk.nvme0n1.in‚ 80.88671875) disk./.usagePercent‚70.08115725563856 disk.dm-1.in‚0.109375 disk.rbd0.in‚0 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚13622.75537109375( proc.memory.availableMB‚ 1730.6484375 proc.cpu.threads‚17 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.dm-0.in‚ 80.88671875) proc.memory.percent‚17.64359474182129 disk.sda.in‚0.109375 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.sda3.out‚ 102.9609375 disk.rbd2.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.sda2.in‚0 disk.sda2.out‚0# disk.dm-0.out‚13622.75537109375 network.sent‚ 5.480311e+06 network.recv‚875656 disk.rbd1.in‚0& disk.nvme0n1.out‚13622.75537109375$ memory_percent‚89.43696022033691" proc.memory.rssMB‚ 2890.7265625% disk./.usageGB‚1044.2598876953125 disk.dm-1.out‚ 102.9609375 cpu‚25.274344073611843 disk.rbd1.out‚ 1.2890625! disk.rbd2.out‚241.44677734375 disk.sda.out‚ 102.9609375 disk.sda1.out‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.1093758±ç‘ï‚(:† ãââÔæ¸Õu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.155' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.831 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0~Û„Kð‚(:À ãââÔ¶ŠÀx cpu‚25.276649927880865 disk.rbd2.in‚0 disk.sda.out‚ 103.58203125# disk.dm-0.out‚13721.08740234375$ memory_percent‚89.44001197814941 disk.rbd1.out‚ 1.2890625 disk.rbd0.out‚0.078125 disk.dm-0.in‚ 80.88671875) disk./.usagePercent‚70.08115725563856 disk.rbd1.in‚0 disk.sda1.in‚0( proc.memory.availableMB‚ 1730.1484375 disk.drbd1777.in‚0 disk.rbd3.out‚ 1.44140625) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17% disk./.usageGB‚1044.2598876953125 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚13721.08740234375 disk.sda2.out‚0" proc.memory.rssMB‚ 2890.7265625 disk.sda3.out‚ 103.58203125& disk.nvme0n1.out‚13721.08740234375 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 network.sent‚ 5.486766e+06! disk.rbd2.out‚241.50927734375 disk.nvme0n1.in‚ 80.88671875 disk.sda2.in‚0 disk.rbd3.in‚0 network.recv‚880438! disk.nvme0c0n1.in‚ 80.88671875 disk.dm-1.out‚ 103.58203125E@g|2ñ- _wandbruntime‚2543  _runtime‚2543;|CÅò¿ & lr-AdamW/pg1‚0.0011150847784528518 & lr-AdamW/pg2‚0.0011150847784528518 trainer/global_step‚2799 # _timestamp‚1.78839178740213e+09 _runtime‚2543.158193554 _step‚224ÏYÑØÊóÄ & lr-AdamW/pg2‚0.0011150847784528518 trainer/global_step‚2799 # _timestamp‚1.78839178740213e+09 _runtime‚2543.158193554 _step‚224 & lr-AdamW/pg1‚0.0011150847784528518àÙÙÌ£2ô- _wandbruntime‚2543  _runtime‚2543‹0>è'õâO KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2415244934656 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚47844972 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _runtime‚2543.162050061 _step‚225 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2301157330944 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚190163349042176 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚47844974 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚190163135823872 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚189362732835328 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚47844974 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2288500108940 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚29294731 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 107451186 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚190163349042176 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚187861978492928 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚185627068020364 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚183338567911424 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚187861995532288 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2288308112400 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚59628812 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚191777712837632 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚191777977769984 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚183338550872064 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚533 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚44252890 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚185626858984464 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 trainer/global_step‚2799 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 107473786 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚190163135823872 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47844972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14958159 % _timestamp‚1.7883917874033587e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14957626 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2415004905984 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581XÌW²í'öçO R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚190163135823872 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚189362732835328 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚47844974 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2288500108940 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚29294731 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 107451186 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚190163349042176 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚187861978492928 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚185627068020364 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚183338567911424 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚187861995532288 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2288308112400 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚59628812 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚191777712837632 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚191777977769984 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚183338550872064 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚533 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚44252890 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚185626858984464 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 trainer/global_step‚2799 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 107473786 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚190163135823872 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47844972 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14958159 % _timestamp‚1.7883917874033587e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14957626 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2415004905984 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2415244934656 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚47844972 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _runtime‚2543.162050061 _step‚225 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2301157330944 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚190163349042176 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚47844974 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598á­®Ué2÷- _wandbruntime‚2543  _runtime‚2543B‡¯y»øëM WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚187927017529856 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚510 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2302151913984 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2301955721728 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14963379 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2415847981056 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 107488578 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚59627086 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚185691252251988 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2301955721728 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 107488578 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚44268144 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚47861494 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚183402150256640 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2289293992096 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚44267631 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚189428233427968 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 107511204 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚190229186483200 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚185691461288096 _step‚226 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _runtime‚2543.956898711 trainer/global_step‚2799 E?DeviceStatsMoni)o X6tor.on_train_batch_end/active.large_pool.current‚2 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚190229186483200 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 $ _timestamp‚1.788391788199752e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚191844342133760 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚47861494 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚187927034569216 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚47861492 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚59627086 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚183402167296000 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚59649710 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 107511204 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 107511204 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚190229186483200 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚185691461288096 _step‚226 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 _runtime‚2543.956898711 trainer/global_step‚2799 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚190229186483200 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 $ _timestamp‚1.788391788199752e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚191844342133760 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚47861494 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚187927034569216 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚47861492 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚59627086 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚183402167296000 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚59649710 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 107511204 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2301955721728 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14963379 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2415847981056 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 107488578 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚59627086 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚185691252251988 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2301955721728 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 107488578 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚44268144 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚47861494 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚183402150256640 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2289293992096 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚44267631â/Z—2ú- _wandbruntime‚2543  _runtime‚2543“ÕO![ûÕ /loss/FRt3/assignment_loss‚3.230703175907404 /loss/FBt1/detection_loss‚2.6809306144714355 /loss/FBt2/detection_loss‚0.5851786971943719 0loss/SRqqt1/detection_loss‚2.038599378885107 /loss/FBt3/detection_loss‚2.6583264827728272 %loss/total_loss‚46.33516244982395 1loss/FBt1/assignment_loss‚0.06355307652162544 1loss/SRqqt2/assignment_loss‚1.348581619605422 1loss/FBt3/assignment_loss‚0.12715794207062572 trainer/global_step‚2799 0loss/FRt1/assignment_loss‚3.3450515872829563 2loss/SRqqt1/assignment_loss‚1.8645334169664916 1loss/FBt4/assignment_loss‚0.19553238047390348 /loss/FRt3/detection_loss‚2.7109811100321757 /loss/FRt4/detection_loss‚2.6160146903682064 1loss/SRqqt4/detection_loss‚2.4821018672758535 % _timestamp‚1.7883917882022026e+09 epoch‚4 .loss/FRt4/assignment_loss‚4.37279373794407 1loss/FBt2/assignment_loss‚0.14053702540721694 .loss/FRt2/detection_loss‚1.914659416795462 1loss/SRqqt3/detection_loss‚1.9775653896813696 _runtime‚2543.958033139 _step‚227 0loss/FRt2/assignment_loss‚1.9717593556162956 1loss/SRqqt4/assignment_loss‚1.937037683062015 2loss/SRqqt3/assignment_loss‚1.7824869334618463 /loss/FRt1/detection_loss‚2.2694416776463227 1loss/SRqqt2/detection_loss‚1.0017645978927612 .loss/FBt4/detection_loss‚3.019870592488183©„ô`üÚ 2loss/SRqqt3/assignment_loss‚1.7824869334618463 /loss/FRt1/detection_loss‚2.2694416776463227 1loss/SRqqt2/detection_loss‚1.0017645978927612 .loss/FBt4/detection_loss‚3.019870592488183 /loss/FRt3/assignment_loss‚3.230703175907404 /loss/FBt1/detection_loss‚2.6809306144714355 /loss/FBt2/detection_loss‚0.5851786971943719 0loss/SRqqt1/detection_loss‚2.038599378885107 /loss/FBt3/detection_loss‚2.6583264827728272 %loss/total_loss‚46.33516244982395 1loss/FBt1/assignment_loss‚0.06355307652162544 1loss/SRqqt2/assignment_loss‚1.348581619605422 1loss/FBt3/assignment_loss‚0.12715794207062572 trainer/global_step‚2799 0loss/FRt1/assignment_loss‚3.3450515872829563 2loss/SRqqt1/assignment_loss‚1.8645334169664916 1loss/FBt4/assignment_loss‚0.19553238047390348 /loss/FRt3/detection_loss‚2.7109811100321757 /loss/FRt4/detection_loss‚2.6160146903682064 1loss/SRqqt4/detection_loss‚2.4821018672758535 % _timestamp‚1.7883917882022026e+09 epoch‚4 .loss/FRt4/assignment_loss‚4.37279373794407 1loss/FBt2/assignment_loss‚0.14053702540721694 .loss/FRt2/detection_loss‚1.914659416795462 1loss/SRqqt3/detection_loss‚1.9775653896813696 _runtime‚2543.958033139 _step‚227 0loss/FRt2/assignment_loss‚1.9717593556162956 1loss/SRqqt4/assignment_loss‚1.937037683062015ãÞ%R‚‘ý‚(:† òââÔÃúv gpu.0.gpu‚32.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚95.467' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.0934 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ×Ûv5þ‚(:ª òââÔÿˆôx disk.nvme0n1.out‚ 13721.5625 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.sda2.out‚0' proc.memory.availableMB‚ 1730.109375 network.sent‚ 5.492109e+06% disk./.usageGB‚1044.2598838806152" proc.memory.rssMB‚ 2890.7265625 disk.dm-0.out‚ 13721.5625 disk.rbd1.out‚ 1.2890625 disk.rbd2.in‚0! disk.rbd2.out‚241.51318359375 disk.sda.in‚0.109375 disk.sda.out‚ 104.05859375 disk.rbd0.in‚0 disk.dm-1.out‚ 104.05859375 disk.rbd3.in‚0 disk.sda1.in‚0! disk.nvme0c0n1.out‚ 13721.5625 disk.sda2.in‚0 cpu‚25.278751517409866 network.recv‚883906 disk.rbd1.in‚0 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.dm-0.in‚ 80.88671875 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.sda3.out‚ 104.05859375) disk./.usagePercent‚70.08115699963102 disk.nvme0n1.in‚ 80.88671875 disk.dm-1.in‚0.109375) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 80.88671875$ memory_percent‚89.44025039672852½Œ–ÿ‚(:’ ãâÔ¸‘Çu gpu.0.gpu‚45.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.058' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.011599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0X“ªH€‚(:½ ãâÔÉì¬y( disk.nvme0c0n1.out‚13819.89208984375 disk.sda2.in‚0 disk.sda3.in‚0.109375 disk.dm-1.out‚ 104.81640625 disk.drbd1777.out‚0 disk.dm-0.in‚ 80.88671875 disk.rbd1.in‚0 disk.rbd1.out‚ 1.2890625! disk.rbd2.out‚241.57568359375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 80.88671875 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.in‚0) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17 disk.sda1.out‚0' proc.memory.availableMB‚ 1730.359375 network.sent‚ 5.577989e+06% disk./.usageGB‚1044.2598838806152 disk.rbd2.in‚0 disk.sda1.in‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.out‚ 104.81640625$ memory_percent‚89.43872451782227" proc.memory.rssMB‚ 2890.7265625& disk.nvme0n1.out‚13819.89208984375 disk.rbd0.out‚0.078125 disk.sda2.out‚0# disk.dm-0.out‚13819.89208984375 cpu‚25.2807355768186 network.recv‚888508 disk.sda.in‚0.109375 disk.dm-1.in‚0.109375) disk./.usagePercent‚70.08115699963102 disk.sda.out‚ 104.81640625ªŸÂ‚(:’ ãâÔ¡Š·u gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.719' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.543799999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±iÞ8‚‚(:­ ãâÔÒàˆx disk.sda3.in‚0.109375 disk.sda3.out‚ 105.18359375 disk.dm-1.out‚ 105.18359375% disk./.usageGB‚1044.2598838806152 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 disk.rbd1.in‚0! disk.nvme0n1.out‚ 13918.265625$ memory_percent‚89.43862915039062# disk.nvme0c0n1.out‚ 13918.265625 disk.rbd1.out‚ 1.2890625 disk.sda1.in‚0) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚0.109375 cpu‚25.282823998299534 network.recv‚893324) disk./.usagePercent‚70.08115699963102 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.sda.out‚ 105.18359375$ proc.memory.availableMB‚1730.375 network.sent‚ 5.584183e+06 disk.dm-0.in‚ 80.88671875 disk.dm-0.out‚ 13918.265625 disk.sda.in‚0.109375 disk.rbd0.in‚0" proc.memory.rssMB‚ 2890.7265625 disk.rbd2.in‚0! disk.rbd2.out‚241.57568359375 disk.nvme0n1.in‚ 80.88671875! disk.nvme0c0n1.in‚ 80.88671875\’Ù2ƒ- _wandbruntime‚2585  _runtime‚2585¥ÞIKÇ„Á & lr-AdamW/pg1‚0.0010912765402283752 & lr-AdamW/pg2‚0.0010912765402283752 trainer/global_step‚2849 % _timestamp‚1.7883918295975237e+09 _runtime‚2585.353502923 _step‚228Þ-½RÌ…Æ _runtime‚2585.353502923 _step‚228 & lr-AdamW/pg1‚0.0010912765402283752 & lr-AdamW/pg2‚0.0010912765402283752 trainer/global_step‚2849 % _timestamp‚1.7883918295975237e+09ä™Å92†- _wandbruntime‚2585  _runtime‚2585º¯Ò»&‡áO PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚60653166 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15215449 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚192638982747648 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 109324138 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚48670974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2341104698368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚193455035037696 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚29796428 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚193455035037696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚193455248256000 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2341300877312 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2341300877312 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _runtime‚ 2585.35803349 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚191113947378688 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15214937 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚48670972 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚188846549532672 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚48670974 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚192639007651328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚191113930339328 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 trainer/global_step‚2849 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2457228932096 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚188846758568572 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚195095973748224 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚186518537140224 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 109346738 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚48670972 _step‚229 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚60653166 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2341104698368 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚n ×ç1130 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚45011880 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚60675764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2?×Âjì'ˆæO JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚195095973748224 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚186518537140224 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 109346738 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚48670972 _step‚229 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚60653166 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2341104698368 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚45011880 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚60675764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚60653166 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15215449 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚192638982747648 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 109324138 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚48670974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2341104698368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚193455035037696 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚29796428 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚193455035037696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚193455248256000 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2341300877312 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2341300877312 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 _runtime‚ 2585.35803349 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚191113947378688 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15214937 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚48670972 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚188846549532672 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚48670974 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚192639007651328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚191113930339328 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 trainer/global_step‚2849 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2457228932096 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚188846758568572 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304å u‹2‰- _wandbruntime‚2586  _runtime‚2586B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚48687492 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚60696662 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚60674038 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2328823314756 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2341903089152 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 109361530 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2341903089152 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2342099281408 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2457828537344 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚45026641 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚193521085697024 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚48687494 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15220179 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚230 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 109384156 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚29806465 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 109384156 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 109361530 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚192704508243968 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚45027167 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚29806462 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚192704533147648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2329015311504 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824eEZe÷&‹ñM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚191178986415616 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚48687492 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚60696662 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚60674038 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2328823314756 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2341903089152 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 109361530 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2341903089152 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2342099281408 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚526 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2457828537344 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚45026641 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚193521085697024 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚48687494 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15220179 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚230 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 109384156 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚29806465 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 109384156 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 109361530 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚192704508243968 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚45027167 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚29806462 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚192704533147648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2329015311504 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚193521085697024 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚193520872465408 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚195162603797504 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚186582119485440 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 % _timestamp‚1.7883918303510938e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15220702 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚188911151836304 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 trainer/global_step‚2849æì¿T2Œ- _wandbruntime‚2586  _runtime‚2586ƒŠè\Ö 2loss/SRqqt1/assignment_loss‚1.8352021828696534 0loss/FBt1/assignment_loss‚0.1455851766950218 0loss/FBt3/assignment_loss‚0.4288908700746106 /loss/FRt2/detection_loss‚1.8146712478412235 /loss/FRt3/detection_loss‚2.5070808191191065 .loss/FBt1/detection_loss‚3.341466039419174 % _timestamp‚1.7883918303533537e+09 /loss/FRt1/assignment_loss‚3.424842653971798 0loss/FRt2/assignment_loss‚1.9751582244168158 /loss/FRt3/assignment_loss‚3.705424280329184 0loss/FBt4/assignment_loss‚0.3029832930420525 /loss/FRt1/detection_loss‚2.2283313035402657 /loss/FRt4/detection_loss‚2.2789252538430063 0loss/SRqqt4/detection_loss‚2.220674529105802 2loss/SRqqt2/assignment_loss‚1.4245757487164263 1loss/SRqqt3/detection_loss‚2.2995283498269754 /loss/FBt3/detection_loss‚2.3345211800187826 1loss/SRqqt1/detection_loss‚1.8135707452222034 1loss/SRqqt2/detection_loss‚1.0593529947321205 epoch‚4 0loss/FRt4/assignment_loss‚4.1615506611372295 2loss/SRqqt4/assignment_loss‚2.0501513276484946 _runtime‚2586.109249012 2loss/SRqqt3/assignment_loss‚1.7444624926258878 &loss/total_loss‚46.220136850447325 trainer/global_step‚2849 _step‚231 1loss/FBt2/assignment_loss‚0.09821903511399496 .loss/FBt2/detection_loss‚0.573248769649688 /loss/FBt4/detection_loss‚2.4517196714878082ŽPÐøLŽÛ &loss/total_loss‚46.220136850447325 trainer/global_step‚2849 _step‚231 1loss/FBt2/assignment_loss‚0.09821903511399496 .loss/FBt2/detection_loss‚0.573248769649688 /loss/FBt4/detection_loss‚2.4517196714878082 2loss/SRqqt1/assignment_loss‚1.8352021828696534 0loss/FBt1/assignment_loss‚0.1455851766950218 0loss/FBt3/assignment_loss‚0.4288908700746106 /loss/FRt2/detection_loss‚1.8146712478412235 /loss/FRt3/detection_loss‚2.5070808191191065 .loss/FBt1/detection_loss‚3.341466039419174 % _timestamp‚1.7883918303533537e+09 /loss/FRt•TÖI1/assignment_loss‚3.424842653971798 0loss/FRt2/assignment_loss‚1.9751582244168158 /loss/FRt3/assignment_loss‚3.705424280329184 0loss/FBt4/assignment_loss‚0.3029832930420525 /loss/FRt1/detection_loss‚2.2283313035402657 /loss/FRt4/detection_loss‚2.2789252538430063 0loss/SRqqt4/detection_loss‚2.220674529105802 2loss/SRqqt2/assignment_loss‚1.4245757487164263 1loss/SRqqt3/detection_loss‚2.2995283498269754 /loss/FBt3/detection_loss‚2.3345211800187826 1loss/SRqqt1/detection_loss‚1.8135707452222034 1loss/SRqqt2/detection_loss‚1.0593529947321205 epoch‚4 0loss/FRt4/assignment_loss‚4.1615506611372295 2loss/SRqqt4/assignment_loss‚2.0501513276484946 _runtime‚2586.109249012 2loss/SRqqt3/assignment_loss‚1.7444624926258878çB~>¶‚(:’ ŸãâÔõÝØu gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.136' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.027199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0h‡$dE‚(:º ŸãâÔí¯x network.recv‚898254% disk./.usageGB‚1044.2600059509277# disk.dm-0.out‚13919.09814453125 disk.nvme0n1.in‚ 80.88671875 disk.sda2.in‚0 cpu‚25.28504337052164 disk.rbd1.in‚0 disk.sda.out‚ 105.7421875) proc.memory.percent‚17.64359474182129 network.sent‚ 5.59074e+06 disk.dm-1.in‚0.109375 proc.cpu.threads‚17 disk.sda3.out‚ 105.7421875 disk.rbd2.in‚0 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚13919.09814453125) disk./.usagePercent‚70.08116519187212 disk.rbd1.out‚ 1.2890625! disk.rbd2.out‚242.71240234375 disk.sda1.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 1.44140625 disk.dm-1.out‚ 105.7421875 disk.sda.in‚0.109375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0" proc.memory.rssMB‚ 2890.7265625 disk.dm-0.in‚ 80.88671875 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 80.88671875 disk.sda2.out‚0$ memory_percent‚89.43796157836914' proc.memory.availableMB‚ 1730.484375 disk.sda3.in‚0.109375& disk.nvme0n1.out‚13919.09814453125mµÜÀœ‘‚(:‘ ®ãâÔ‰Û¨u gpu.0.gpu‚23.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚154.68' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.936000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0WUÖPD’‚(:¹ ®ãâÔ²•òw disk.sda.out‚ 106.546875 disk.sda1.out‚0 disk.drbd1777.in‚0 network.sent‚ 5.677168e+06 disk.drbd1777.out‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.sda3.out‚ 106.546875 disk.nvme0n1.in‚ 80.88671875 disk.sda2.out‚0 disk.dm-0.in‚ 80.88671875# disk.dm-0.out‚14017.40673828125 disk.rbd1.in‚0 disk.rbd2.in‚0 proc.cpu.threads‚17# disk./.usageGB‚1044.26025390625$ memory_percent‚89.43803310394287 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.in‚0 disk.rbd1.out‚ 1.2890625 disk.rbd0.in‚0! disk.rbd2.out‚243.82177734375) proc.memory.availableMB‚ 1730.47265625 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125( disk.nvme0c0n1.out‚14017.40673828125 disk.dm-1.in‚0.109375" proc.memory.rssMB‚ 2890.7265625) disk./.usagePercent‚70.08118183236184& disk.nvme0n1.out‚14017.40673828125 disk.sda2.in‚0 disk.dm-1.out‚ 106.546875) proc.memory.percent‚17.64359474182129 cpu‚25.287069601202532 network.recv‚9035691L¾À“‚(:’ ½ãâÔÖ¼Éu gpu.0.gpu‚29.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚154.152' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.830399999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0DÊTA”‚(:¶ ½ãâÔÉ·x% disk./.usageGB‚1044.2602577209473 disk.rbd3.in‚0 disk.rbd1.out‚1.375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.sda1.in‚0) disk./.usagePercent‚70.08118208836937 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 108.44140625 disk.dm-1.out‚ 108.44140625! disk.rbd2.out‚247.18896484375 disk.rbd0.out‚0.078125 network.sent‚ 5.682569e+06 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 80.88671875! disk.dm-0.out‚14022.259765625 disk.sda.out‚ 108.44140625) proc.memory.percent‚17.64359474182129 disk.drbd1777.in‚0 disk.dm-0.in‚ 80.88671875" proc.memory.rssMB‚ 2890.7265625 cpu‚25.289145373941395 disk.sda.in‚0.109375 disk.rbd2.in‚0$ disk.nvme0n1.out‚14022.259765625 network.recv‚907405 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.rbd1.in‚0 disk.nvme0n1.in‚ 80.88671875$ memory_percent‚89.43953514099121 proc.cpu.threads‚17& disk.nvme0c0n1.out‚14022.259765625 disk.dm-1.in‚0.109375( proc.memory.availableMB‚ 1730.2265625¨Š8ð2•- _wandbruntime‚2627  _runtime‚2627Èt Å–¿ & lr-AdamW/pg2‚0.0010672746140526533 trainer/global_step‚2899 # _timestamp‚1.78839187163641e+09 _runtime‚2627.392317518 _step‚232 & lr-AdamW/pg1‚0.0010672746140526533÷*C™Ê—Ä trainer/global_step‚2899 # _timestamp‚1.78839187163641e+09 _runtime‚2627.392317518 _step‚232 & lr-AdamW/pg1‚0.0010672746140526533 & lr-AdamW/pg2‚0.0010672746140526533è*~tŽ2˜- _wandbruntime‚2627  _runtime‚2627™ ®è'™âO M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚45771679 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2499364811264 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚192066240080880 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 trainer/global_step‚2899 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚531 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚189698506369024 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚49496972 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚198414647278592 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚49496972 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚194365882185728 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚49496974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚195915257563648 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚196747147469824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15474082 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚61722716 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚196746934251520 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚195915282467328 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2367942747756 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚45772213 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚61700118 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚194365899225088 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚194365899225088 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚194365882185728 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚534 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _runtime‚2627.393608951 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚196747147469824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚30298131 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2499126879744 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2367750751216 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 111219690 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚61700118 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚196746934251520 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2381052065792 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 % _timestamp‚1.7883918716373723e+09 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚49496972 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚194365882185728 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚49496974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚195915257563648 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚196747147469824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15474082 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚61722716 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚196746934251520 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚195915282467328 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2367942747756 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚45772213 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚61700118 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚194365899225088 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚194365899225088 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚194365882185728 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚534 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _runtime‚2627.393608951 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚196747147469824 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚30298131 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2499126879744 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2367750751216 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 111219690 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚61700118 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚196746934251520 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2381052065792 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 % _timestamp‚1.7883918716373723e+09 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 J6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚2628.138458422 _step‚234 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚195980807963648 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2382046648832 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 111257108 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 111257108 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 111234482 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚61720990 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚192130633348404 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚49513494 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚192130842384512 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 111234482 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.a|xáš ll.freed‚196812771679232 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2368544634164 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚61743614 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2368736630912 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚45786774 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚30308165 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚194430938262016 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15479132 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15478612 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚196812984910848 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚198481015201792 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚196812984910848 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚520 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚49513492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚49513492 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚523 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚30308162 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚61743614 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2499977708544 % _timestamp‚1.7883918723820627e+09v„·Ø÷&ñM C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚49513492 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚49513492 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚523 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚30308162 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚61743614 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2499977708544 % _timestamp‚1.7883918723820627e+09 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2381850456576 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚194430921222656 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚2899 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚45787297 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _runtime‚2628.138458422 _step‚234 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚195980807963648 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2382046648832 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 111257108 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 111257108 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 111234482 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚61720990 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚192130633348404 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚49513494 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚192130842384512 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 111234482 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚196812771679232 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2368544634164 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚61743614 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2368736630912 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚45786774 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚30308165 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚194430938262016 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15479132 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15478612 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚196812984910848 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚198481015201792 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚196812984910848 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚520 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0êrýf÷2ž- _wandbruntime‚2628  _runtime‚2628ÔZLXŸÒ /loss/FRt2/assignment_loss‚2.309468454399774 0loss/FRt3/assignment_loss‚3.5057903325015847 2loss/SRqqt3/assignment_loss‚1.9768718833770227 0loss/SRqqt3/detection_loss‚1.793629680068121 epoch‚4 _step‚235 /loss/FRt4/assignment_loss‚4.084282895614361 0loss/FBt4/assignment_loss‚0.2784203106836815 /loss/FRt2/detection_loss‚2.0425884978369226 .loss/FRt3/detection_loss‚2.370960854671218 1loss/SRqqt1/detection_loss‚1.6883109674133636 1loss/SRqqt2/detection_loss‚1.1920899282281214 /loss/FBt2/detection_loss‚0.6484490744672561 1loss/SRqqt2/assignment_loss‚1.380751398893503 0loss/FBt2/assignment_loss‚0.1612133626913383 0loss/SRqqt4/detection_loss‚2.368480301820315 trainer/global_step‚2899 % _timestamp‚1.7883918723840742e+09 /loss/FBt4/detection_loss‚2.7709629791123525 0loss/FRt1/assignment_loss‚3.5268498056464725 0loss/FBt1/assignment_loss‚0.4645491839279809 /loss/FBt3/detection_loss‚2.1971005271462833 2loss/SRqqt1/assignment_loss‚1.5647991470716618 0loss/FBt3/assignment_loss‚0.3672309149037841 /loss/FRt4/detection_loss‚2.4024339345679886 .loss/FBt1/detection_loss‚2.726681093374888 _runtime‚2628.139899738 2loss/SRqqt4/assignment_loss‚1.9425708115100861 /loss/FRt1/detection_loss‚2.5878458766473664 %loss/total_loss‚46.35233221657545JiÃ] × 0loss/FBt3/assignment_loss‚0.3672309149037841 /loss/FRt4/detection_loss‚2.4024339345679886 .loss/FBt1/detection_loss‚2.726681093374888 _runtime‚2628.139899738 2loss/SRqqt4/assignment_loss‚1.9425708115100861 /loss/FRt1/detection_loss‚2.5878458766473664 %loss/total_loss‚46.35233221657545 /loss/FRt2/assignment_loss‚2.309468454399774 0loss/FRt3/assignment_loss‚3.5057903325015847 2loss/SRqqt3/assignment_loss‚1.9768718833770227 0loss/SRqqt3/detection_loss‚1.793629680068121 epoch‚4 _step‚235 /loss/FRt4/assignment_loss‚4.084282895614361 0loss/FBt4/assignment_loss‚0.2784203106836815 /loss/FRt2/detection_loss‚2.0425884978369226 .loss/FRt3/detection_loss‚2.370960854671218 1loss/SRqqt1/detection_loss‚1.6883109674133636 1loss/SRqqt2/detection_loss‚1.1920899282281214 /loss/FBt2/detection_loss‚0.6484490744672561 1loss/SRqqt2/assignment_loss‚1.380751398893503 0loss/FBt2/assignment_loss‚0.1612133626913383 0loss/SRqqt4/detection_loss‚2.368480301820315 trainer/global_step‚2899 % _timestamp‚1.7883918723840742e+09 /loss/FBt4/detection_loss‚2.7709629791123525 0loss/FRt1/assignment_loss‚3.5268498056464725 0loss/FBt1/assignment_loss‚0.4645491839279809 /loss/FBt3/detection_loss‚2.1971005271462833 2loss/SRqqt1/assignment_loss‚1.5647991470716618ë§À%›¡‚(:‚ ÌãâÔ÷µÖu gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚154.5' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.9 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0„¡¸d@¢‚(:µ ÌãâÔåŽy disk.sda1.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.out‚ 1.44140625$ memory_percent‚89.43967819213867 proc.cpu.threads‚17 network.sent‚ 5.688683e+06 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.sda.out‚ 109.7109375 disk.sda2.in‚0 disk.sda3.in‚0.109375' proc.memory.availableMB‚ 1730.203125 disk.sda.in‚0.109375 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.sda3.out‚ 109.7109375" proc.memory.rssMB‚ 2890.7265625% disk.nvme0n1.out‚14296.0634765625 disk.dm-1.in‚0.109375 disk.rbd1.in‚0 disk.nvme0n1.in‚ 80.88671875) proc.memory.percent‚17.64359474182129" disk.dm-0.out‚14296.0634765625 disk.dm-1.out‚ 109.7109375 disk.rbd1.out‚1.4375 disk.rbd2.in‚0! disk.rbd2.out‚252.42724609375 disk.rbd0.in‚0 disk.rbd0.out‚0.078125' disk.nvme0c0n1.out‚14296.0634765625 network.recv‚912076% disk./.usageGB‚1044.8197593688965 disk.sda1.out‚0 disk.dm-0.in‚ 80.88671875 cpu‚25.291188744410167( disk./.usagePercent‚70.1187307134164ý w㣂(:’ ÛãâÔÔ±Óu gpu.0.gpu‚53.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚119.444' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.888800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0fU3¨-¤‚(:¢ ÛãâÔꄵx! disk.nvme0c0n1.in‚ 80.88671875 disk.dm-1.out‚110.5 disk.rbd1.in‚0 cpu‚25.293159445653586 disk.sda1.in‚0 network.recv‚918170% disk./.usageGB‚1045.1321258544922 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.in‚0! disk.dm-0.out‚14402.447265625 proc.cpu.threads‚17 disk.sda3.out‚110.5! disk.rbd2.out‚257.75146484375 disk.sda1.out‚0 disk.rbd0.in‚0( proc.memory.availableMB‚ 1730.1796875 disk.rbd3.out‚ 1.44140625 disk.dm-1.in‚0.109375 disk.nvme0n1.in‚ 80.88671875$ disk.nvme0n1.out‚14402.447265625 disk.sda.out‚110.5 disk.rbd0.out‚0.078125) disk./.usagePercent‚70.13969389035766 disk.rbd1.out‚1.4375 disk.sda.in‚0.109375 disk.sda3.in‚0.109375 network.sent‚ 5.776266e+06 disk.drbd1777.in‚0 disk.drbd1777.out‚0& disk.nvme0c0n1.out‚14402.447265625 disk.dm-0.in‚ 80.88671875 disk.rbd2.in‚0" proc.memory.rssMB‚ 2890.7265625) proc.memory.percent‚17.64359474182129$ memory_percent‚89.43982124328613aD<2¥- _wandbruntime‚2668  _runtime‚2668m³:dŦ¿ _step‚236 % lr-AdamW/pg1‚0.001043099039880322 % lr-AdamW/pg2‚0.001043099039880322 trainer/global_step‚2949 % _timestamp‚1.7883919130397627e+09 _runtime‚2668.795740875IëSù椀 trainer/global_step‚2949 % _timestamp‚1.7883919130397627e+09 _runtime‚2668.795740875 _step‚236 % lr-AdamW/pg1‚0.001043099039880322 % lr-AdamW/pg2‚0.001043099039880322ìã°pf2¨- _wandbruntime‚2668  _runtime‚2668‹ASåè'©âO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚30799828 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2420999433216 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2407664067164 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚62747070 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚46532373 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚199191557283328 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚62769668 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2420999433216 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚50322972 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚197617834032128 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2407472070624 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚46532915 _runtime‚2668.797115005 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2421195612160 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2541328856064 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚195285930629088 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚200038833465344 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚62747070 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚46532373 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚199191557283328 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚62769668 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2420999433216 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes’øîÀ°.all.allocated‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚50322972 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚197617834032128 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2407472070624 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚46532915 _runtime‚2668.797115005 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2421195612160 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2541328856064 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚195285930629088 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚200038833465344 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 J2«- _wandbruntime‚2669  _runtime‚2669d¨9ò&¬ìM KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚197682890108416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚50339492 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚195350532932720 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚50339494 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚50339492 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2408265953572 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚197682873069056 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 113107434 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚199257082779648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚62767942 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚30809862 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚200104884124672 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2421994016256 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2542168183296 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚46547562 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 113107434 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 113130060 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚515 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚195350323896612 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚62790566 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚200104884124672 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271210496 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2542414490112 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15738212 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 _step‚238 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15737700 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚201799497269760 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚192942074982400 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2408457950320 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚50339494 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚30809865 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 % _timestamp‚1.7883919137822826e+09 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 113130060 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚201799226059264 trainer/global_step‚2949 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚197682890108416 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3Y«éa÷&­ñM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚197682890108416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚50339492 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚195350532932720 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚50339494 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚50339492 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2408265953572 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚197682873069056 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 113107434 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚199257082779648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚62767942 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚30809862 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚200104884124672 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2421994016256 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2542168183296 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚46547562 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 113107434 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 113130060 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚515 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚195350323896612 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚62790566 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚200104884124672 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 271210496 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2542414490112 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15738212 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 _step‚238 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15737700 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚201799497269760 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚192942074982400 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2408457950320 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚50339494 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚30809865 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 % _timestamp‚1.7883919137822826e+09 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 113130060 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚201799226059264 trainer/global_step‚2949 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚197682890108416 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28îÛU"á2®- _wandbruntime‚2669  _runtime‚2669}DU~Y¯Ó /loss/FRt2/detection_loss‚1.8334901602092115 &loss/total_loss‚44.641334611689956 _runtime‚2669.539589979 0loss/FRt2/assignment_loss‚1.9239269149574367 /loss/FRt1/detection_loss‚2.3391380739212035 .loss/SRqqt3/detection_loss‚1.8082398477028 /loss/FBt4/detection_loss‚2.5265263285901813 epoch‚4 1loss/FBt2/assignment_loss‚0.12245053132828783 .loss/FBt3/detection_loss‚2.749141311645508 0loss/FBt1/assignment_loss‚0.0166525394576768 trainer/global_step‚2949 /loss/FRt4/assignment_loss‚3.780931036863754 1loss/SRqqt2/assignment_loss‚1.284623535632184 2loss/SRqqt3/assignment_loss‚1.6230011913784832 1loss/FBt3/assignment_loss‚0.20600165630457923 /loss/FRt3/detection_loss‚2.6667547918071874 % _timestamp‚1.7883919137838418e+09 0loss/FRt1/assignment_loss‚3.1232736027240753 /loss/FRt3/assignment_loss‚3.531754915651522 /loss/FBt2/detection_loss‚0.5940240274848683 _step‚239 2loss/SRqqt4/assignment_loss‚1.6215857585009776 .loss/FRt4/detection_loss‚2.644534006937226 0loss/SRqqt2/detection_loss‚1.077385204533736 /loss/FBt1/detection_loss‚2.9261526550565446 1loss/FBt4/assignment_loss‚0.13075329462622498 0loss/SRqqt1/detection_loss‚1.867461679083807 1loss/SRqqt4/detection_loss‚2.4640755005050123 2loss/SRqqt1/assignment_loss‚1.7794560467874683 ~²"^°Ø % _timestamp‚1.7883919137838418e+09 0loss/FRt1/assignment_loss‚3.1232736027240753 /loss/FRt3/assignment_loss‚3.531754915651522 /loss/FBt2/detection_loss‚0.5940240274848683 _step‚239 2loss/SRqqt4/assignment_loss‚1.6215857585009776 .loss/FRt4/detection_loss‚2.644534006937226 0loss/SRqqt2/detection_loss‚1.077385204533736 /loss/FBt1/detection_loss‚2.9261526550565446 1loss/FBt4/assignment_loss‚0.13075329462622498 0loss/SRqqt1/detection_loss‚1.867461679083807 1loss/SRqqt4/detection_loss‚2.4640755005050123 2loss/SRqqt1/assignment_loss‚1.7794560467874683 /loss/FRt2/detection_loss‚1.8334901602092115 &loss/total_loss‚44.641334611689956 _runtime‚2669.539589979 0loss/FRt2/assignment_loss‚1.9239269149574367 /loss/FRt1/detection_loss‚2.3391380739212035 .loss/SRqqt3/detection_loss‚1.8082398477028 /loss/FBt4/detection_loss‚2.5265263285901813 epoch‚4 1loss/FBt2/assignment_loss‚0.12245053132828783 .loss/FBt3/detection_loss‚2.749141311645508 0loss/FBt1/assignment_loss‚0.0166525394576768 trainer/global_step‚2949 /loss/FRt4/assignment_loss‚3.780931036863754 1loss/SRqqt2/assignment_loss‚1.284623535632184 2loss/SRqqt3/assignment_loss‚1.6230011913784832 1loss/FBt3/assignment_loss‚0.20600165630457923 /loss/FRt3/detection_loss‚2.6667547918071874ïÿÈ(Ò±‚(:’ êãâÔ„Èïv gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚124.815' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.962999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Í€€¨C²‚(:¸ êãâÔÒ¦äw$ disk.nvme0n1.out‚14917.068359375 disk.rbd0.in‚0 disk.dm-0.in‚ 80.88671875 network.sent‚ 5.781682e+06 disk.sda3.in‚0.109375 disk.rbd2.in‚0 disk.drbd1777.in‚0 disk.sda2.in‚0 network.recv‚922263% disk./.usageGB‚1044.9638328552246 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.out‚0! disk.dm-0.out‚14917.068359375 proc.cpu.threads‚17 disk.rbd1.out‚ 1.4697265625 disk.nvme0n1.in‚ 80.88671875! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.in‚0) proc.memory.availableMB‚ 1730.19921875) disk./.usagePercent‚70.12839960596958 disk.sda3.out‚ 111.140625 disk.rbd1.in‚0 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125& disk.nvme0c0n1.out‚14917.068359375! disk.rbd2.out‚262.23974609375 disk.sda2.out‚0) proc.memory.percent‚17.64359474182129 cpu‚25.295100706942478 disk.rbd3.out‚ 1.44140625 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 111.140625 disk.sda.out‚ 111.140625" proc.memory.rssMB‚ 2890.7265625$ memory_percent‚89.43970203399658¨ 𼳂(:’ ùãâÔ¥¾u gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.192' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.838400000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0l~äÅ‚´‚(:Á ùãâÔ£Õ­x proc.cpu.threads‚17 network.sent‚ 5.86826e+06! disk.rbd2.out‚269.75634765625 disk.sda.out‚ 111.95703125" proc.memory.rssMB‚ 2890.7265625 disk.rbd3.in‚0 disk.sda3.out‚ 111.95703125 disk.dm-1.out‚ 111.95703125( disk.nvme0c0n1.out‚15025.89013671875 disk.rbd1.in‚0$ disk./.usageGB‚1044.963291168213 disk.rbd3.out‚ 1ºË>Ê.44140625 disk.nvme0n1.in‚ 80.88671875 disk.rbd0.out‚0.078125 cpu‚25.297088110656478 disk.drbd1777.out‚0# disk.dm-0.out‚15025.89013671875 disk.sda3.in‚0.109375 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda2.in‚0 network.recv‚927280 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 1.4697265625) disk./.usagePercent‚70.12836325289973 disk.dm-0.in‚ 80.88671875 disk.dm-1.in‚0.109375) proc.memory.percent‚17.64359474182129& disk.nvme0n1.out‚15025.89013671875! disk.nvme0c0n1.in‚ 80.88671875$ memory_percent‚89.43829536437988( proc.memory.availableMB‚ 1730.4296875!2t5µ‚(:’ ˆäâÔÿøÂv gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚35.0 gpu.0.powerWatts‚153.616' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.723200000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‘3ö?¶‚(:´ ˆäâÔÉ®žx disk.rbd1.in‚0& proc.memory.availableMB‚ 1730.34375" proc.memory.rssMB‚ 2890.7265625) disk./.usagePercent‚70.38841775424129 disk.rbd3.in‚0 network.sent‚ 5.874267e+06 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.dm-1.in‚0.109375 cpu‚25.298977875085733 network.recv‚932043 disk.rbd0.in‚0( disk.nvme0c0n1.out‚15164.19091796875 disk.sda2.in‚0 disk.drbd1777.out‚0 disk.dm-0.in‚ 80.88671875# disk.dm-0.out‚15164.19091796875 disk.rbd2.in‚0& disk.nvme0n1.out‚15164.19091796875 disk.sda.in‚0.109375 disk.sda1.in‚0 disk.sda1.out‚0! disk.rbd2.out‚277.10009765625) proc.memory.percent‚17.64359474182129 proc.cpu.threads‚17 disk.nvme0n1.in‚ 80.88671875 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.rbd1.out‚ 1.53271484375$ disk./.usageGB‚1048.838291168213 disk.dm-1.out‚112.4375# memory_percent‚89.4388198852539 disk.sda.out‚112.4375 disk.sda3.out‚112.4375“€æ2·- _wandbruntime‚2710  _runtime‚2710Á¢§Ã¸½ _runtime‚2710.203598107 _step‚240 $ lr-AdamW/pg1‚0.00101877000265013 $ lr-AdamW/pg2‚0.00101877000265013 trainer/global_step‚2999 % _timestamp‚1.7883919544475632e+09 c·È¹Â _runtime‚2710.203598107 _step‚240 $ lr-AdamW/pg1‚0.00101877000265013 $ lr-AdamW/pg2‚0.00101877000265013 trainer/global_step‚2999 % _timestamp‚1.7883919544475632e+09ðtB˜2º- _wandbruntime‚2710  _runtime‚2710XÙ£ç'»áO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15991196 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚205051185128448 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚200869785878528 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 114942994 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚31301531 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚200869785878528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 trainer/global_step‚2999 _step‚241 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2461142979584 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2460946800640 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚200869802917888 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15991734 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚47292724 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚31301528 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚63794022 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚203330945897472 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2461142979584 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚203330732679168 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2447385386572 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _runtime‚2710.207016107 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2447193390032 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚538 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2583613767168 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 $ _timestamp‚1.788391954448677e+09 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2460946800640 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚196058427787264 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2583377932800 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚205051445866496 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚63816620 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚541 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368ÁøNì'¼æO N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 114965594 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚51148974 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚51148974 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚200869802917888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚198505621177296 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚203330732679168 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2447385386572 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _runtime‚2710.207016107 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2447193390032 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚538 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2583613767168 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 $ _timestamp‚1.788391954448677e+09 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2460946800640 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚196058427787264 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2583377932800 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚205051445866496 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚63816620 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚541 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15991196 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚205051185128448 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚200869785878528 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 114942994 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚31301531 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚200869785878528 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 trainer/global_step‚2999 _step‚241 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2461142979584 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2460946800640 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚200869802917888 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15991734 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚47292724 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚31301528 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚63794022 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚203330945897472 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2461142979584 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚47307824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚51165494 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚200934824915456 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 trainer/global_step‚2999 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚63837518 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2461745191424 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚200934841954816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15996769 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚202533357595648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2584226840576 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2461941383680 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚63814894 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚200934824915456 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 114980386 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2584468953088 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚196122044211200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 115003012 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚47308334 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E=ƒ0?¶ÙDeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚203396570106880 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0*=<÷&¿ñM OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚47308334 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚203396570106880 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚205117826548736 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚198570014444820 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 % _timestamp‚1.7883919551943433e+09 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 115003012 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 114980386 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2461941383680 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚47307824 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚51165494 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚200934824915456 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 trainer/global_step‚2999 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚63837518 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2461745191424 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚200934841954816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15996769 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚202533357595648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2584226840576 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2461941383680 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚63814894 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚200934824915456 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 114980386 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2584468953088 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚196122044211200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 115003012ò™X¹2À- _wandbruntime‚2710  _runtime‚2710ñ· \ÁÖ /loss/FRt4/detection_loss‚2.5111976054153944 1loss/SRqqt1/detection_loss‚1.7836504389521897 /loss/FBt4/detection_loss‚2.7676463597699215 %loss/total_loss‚45.61167417245841 % _timestamp‚1.7883919551964202e+09 1loss/FBt2/assignment_loss‚0.14938959073475183 /loss/FRt3/detection_loss‚2.2883678939938545 /loss/FBt1/detection_loss‚2.5883273124694823 _runtime‚2710.952215752 /loss/FRt2/detection_loss‚1.6619952659843102 0loss/SRqqt4/detection_loss‚2.339634266578489 /loss/FBt2/detection_loss‚0.6004068281335038 1loss/FBt3/assignment_loss‚0.17446600424591452 .loss/FBt3/detection_loss‚2.943481981754303 0loss/FRt1/assignment_loss‚3.5614500479264692 0loss/FRt2/assignment_loss‚2.0926525963927216 1loss/SRqqt2/detection_loss‚0.9933727978522541 epoch‚4 /loss/FRt4/assignment_loss‚4.012616988859679 2loss/SRqqt2/assignment_loss‚1.2991694277934445 2loss/SRqqt3/assignment_loss‚1.7996117965348306 _step‚243 trainer/global_step‚2999 2loss/SRqqt4/assignment_loss‚1.9603336658328772 0loss/FBt1/assignment_loss‚0.3340820825813959 /loss/FBt4/assignment_loss‚0.105911928382212 .loss/FRt1/detection_loss‚2.615663202552052 1loss/SRqqt3/detection_loss‚1.9330519500278658 0loss/FRt3/assignment_loss‚3.3822915029525755 2loss/SRqqt1/assignment_loss‚1.7129026367379132O?ì5aÂÛ 0loss/FRt1/assignment_loss‚3.5614500479264692 0loss/FRt2/assignment_loss‚2.0926525963927216 1loss/SRqqt2/detection_loss‚0.9933727978522541 epoch‚4 /loss/FRt4/assignment_loss‚4.012616988859679 2loss/SRqqt2/assignment_loss‚1.2991694277934445 2loss/SRqqt3/assignment_loss‚1.7996117965348306 _step‚243 trainer/global_step‚2999 2loss/SRqqt4/assignment_loss‚1.9603336658328772 0loss/FBt1/assignment_loss‚0.3340820825813959 /loss/FBt4/assignment_loss‚0.105911928382212 .loss/FRt1/detection_loss‚2.615663202552052 1loss/SRqqt3/detection_loss‚1.9330519500278658 0loss/FRt3/assignment_loss‚3.3822915029525755 2loss/SRqqt1/assignment_loss‚1.7129026367379132 /loss/FRt4/detection_loss‚2.5111976054153944 1loss/SRqqt1/detection_loss‚1.7836504389521897 /loss/FBt4/detection_loss‚2.7676463597699215 %loss/total_loss‚45.61167417245841 % _timestamp‚1.7883919551964202e+09 1loss/FBt2/assignment_loss‚0.14938959073475183 /loss/FRt3/detection_loss‚2.2883678939938545 /loss/FBt1/detection_loss‚2.5883273124694823 _runtime‚2710.952215752 /loss/FRt2/detection_loss‚1.6619952659843102 0loss/SRqqt4/detection_loss‚2.339634266578489 /loss/FBt2/detection_loss‚0.6004068281335038 1loss/FBt3/assignment_loss‚0.17446600424591452 .loss/FBt3/detection_loss‚2.943481981754303óU¥ô˜Ã‚(:’ —äâÔå¹Âu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.957' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.791400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0JÖÿCÄ‚(:¸ —äâÔÞ¹Ôx disk.rbd3.out‚ 1.44140625 disk.sda3.out‚ 113.1484375 disk.rbd2.in‚0 cpu‚25.30092637135245 network.sent‚ 5.880774e+06 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 80.88671875! disk.rbd2.out‚285.75341796875 disk.sda1.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17 disk.dm-0.out‚18148.95703125 disk.dm-1.out‚ 113.1484375# disk.nvme0n1.out‚18148.95703125$ memory_percent‚89.44056034088135) proc.memory.availableMB‚ 1730.05859375) proc.memory.percent‚17.64359474182129 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 80.88671875% disk.nvme0c0n1.out‚18148.95703125 network.recv‚936737% disk./.usageGB‚1048.8382949829102 disk.rbd1.out‚ 1.53271484375 disk.sda.out‚ 113.1484375 disk.sda2.in‚0) disk./.usagePercent‚70.38841801024883 disk.nvme0n1.in‚ 80.88671875 disk.rbd0.in‚0" proc.memory.rssMB‚ 2890.7265625 disk.drbd1777.out‚0»i‡ˆÅ‚(:… ¦äâÔÊÙ«u gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚88.876' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7752 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0êþ?åBÆ‚(:· ¦äâÔ–ðx$ memory_percent‚89.44485187530518 disk.dm-0.in‚ 80.88671875 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375! disk.rbd2.out‚293.21826171875 network.sent‚ 5.967333e+06 disk.nvme0n1.in‚ 80.88671875 disk.rbd2.in‚0 network.recv‚941853 disk.sda.in‚0.109375 disk.sda.out‚ 113.61328125 disk.sda2.in‚0 disk.rbd3.in‚0$ disk./.usageGB‚1048.838279724121 disk.rbd3.out‚ 1.44140625" proc.memory.rssMB‚ 2891.5859375 cpu‚25.308679206684005 disk.drbd1777.out‚0$ disk.nvme0c0n1.out‚ 18247.2890625 disk.sda2.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 1.595703125* proc.memory.percent‚17.648839950561523" disk.nvme0n1.out‚ 18247.2890625 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.dm-1.out‚ 113.61328125) proc.memory.availableMB‚ 1729.35546875) disk./.usagePercent‚70.38841698621869 proc.cpu.threads‚17 disk.drbd1777.in‚0 disk.sda1.in‚0 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.dm-0.out‚ 18247.2890625 disk.sda3.out‚ 113.61328125I?'¥›Ç‚(: µäâÔûä§w gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.683' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.736600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0pÂÒtEÈ‚(:º µäâÔÄ›ú{ disk.sda3.out‚ 116.44921875 disk.rbd1.out‚ 1.595703125 disk.rbd2.out‚ 300.84375$ memory_percent‚89.89958763122559 network.sent‚ 5.972796e+06 disk.rbd0.in‚0' disk.nvme0c0n1.out‚19396.3427734375" disk.dm-0.out‚19396.3427734375" proc.memory.rssMB‚ 2966.3984375 proc.cpu.threads‚17) disk./.usagePercent‚70.44595595957685( proc.memory.availableMB‚ 1654.8515625! disk.nvme0c0n1.in‚ 80.88671875 disk.dm-1.out‚ 116.44921875 disk.nvme0n1.in‚ 80.88671875 disk.rbd0.out‚0.078125 disk.rbd3.in‚0 disk.sda2.in‚0 disk.rbd1.in‚0% disk.nvme0n1.out‚19396.3427734375 cpu‚25.336564536861477% disk./.usageGB‚1049.6956520080566 disk.rbd3.out‚ 1.44140625 disk.sda3.in‚0.109375 disk.dm-1.in‚0.109375 disk.rbd2.in‚0 disk.sda.out‚ 116.44921875 disk.sda1.out‚0* proc.memory.percent‚18.105459213256836 disk.drbd1777.in‚0 disk.dm-0.in‚ 80.88671875 disk.sda.in‚0.109375 disk.sda1.in‚0 network.recv‚945574 disk.drbd1777.out‚0 disk.sda2.out‚0ç!b9É‚(:’ ÄäâÔµÎèu gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚161.877' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.375400000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“Ó‚ÿCÊ‚(:¸ ÄäâÔ¡›ów disk.dm-1.in‚0.109375 disk.nvme0n1.in‚ 80.88671875! disk.nvme0c0n1.in‚ 80.88671875 disk.sda2.in‚0 cpu‚25.36603569473492 disk.rbd3.in‚0 disk.dm-0.in‚ 80.88671875& disk.nvme0n1.out‚19499.14404296875 disk.sda.out‚ 117.8515625 network.recv‚950471 disk.rbd2.in‚0 disk.sda1.out‚0 disk.sda3.in‚0.109375 disk.sda3.out‚ 117.8515625* proc.memory.percent‚17.653679847717285$ disk./.usageGB‚1051.296371459961 disk.rbd1.in‚0 disk.rbd2.out‚308.375) disk./.usagePercent‚70.55338158508724( disk.nvme0c0n1.out‚19499.14404296875 disk.rbd3.out‚ 1.44140625# disk.dm-0.out‚19499.14404296875 disk.rbd1.out‚ 1.595703125 disk.sda1.in‚0 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.sda.in‚0.109375 disk.rbd0.in‚0 disk.drbd1777.in‚0$ memory_percent‚89.44947719573975# proc.memory.rssMB‚ 2892.37890625 proc.cpu.threads‚17 network.sent‚ 5.978997e+06 disk.dm-1.out‚ 117.8515625 disk.drbd1777.out‚0) proc.memory.availableMB‚ 1728.59765625óÏ ƒ2Ë- _wandbruntime‚2774  _runtime‚2774\QrÞ44̧ & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.12745097279548645 jet/accuracy_5_of_5‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_1_of_7‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4028061032295227 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.33523640036582947 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26234322786331177 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 particle/accuracy_3_of_5‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1641712188720703 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2977273166179657 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.27232903242111206 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11951180547475815 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.3121572732925415 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 particle/accuracy_2_of_6‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 (particle/f_score‚0.38903990387916565 particle/accuracy_4_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3014635443687439 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.31127551198005676 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 _runtime‚2774.852318694 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.01558668166399002 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3241167366504669 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3110406994819641 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9943825602531433 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.15324676036834717 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4320240318775177 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9964770674705505 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2777777910232544 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1308000534772873 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36947524547576904 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6447368264198303 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9019157886505127 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19838950037956238 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9987443089485168 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6622806787490845 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_4_of_6‚0.04385964944958687 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.007942832075059414 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_6‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13123808801174164 -jet/accuracy_4_of_4‚0.0021601382177323103 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 particle/accuracy_6_of_7‚1 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 _step‚244 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9953917264938354 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.08967042714357376 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986286759376526 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.33832475543022156 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.19949494302272797 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.32651278376579285 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3582138419151306 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 *loss/val_total_loss‚46.118554231121394 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07467762380838394 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016651051118969917 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3445245325565338 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08507245033979416 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31680843234062195 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.46666666865348816 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 *particle/sensitivity‚0.304080069065094 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.004597188904881477 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3592592775821686 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.02251780591905117 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4335184693336487 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2747695744037628 jet/accuracy_3_of_7‚0 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13207842409610748 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.32073697447776794 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.011290322057902813 particle/accuracy_4_of_4‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1388888955116272 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9912092685699463 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15533789992332458 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2668386399745941 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.33098095655441284 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2859530746936798 6 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.162566140294075 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.3499999940395355 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.15747852623462677 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 trainer/global_step‚3019 8"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.382296621799469 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0033064514864236116 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9884257912635803 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11594203114509583 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15768124163150787 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08507245033979416 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 *jet/accuracy_3_of_5‚0.0955095961689949 particle/accuracy_2_of_4‚1 particle/accuracy_4_of_7‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9926327466964722 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.14764520525932312 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_2_of_2‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3592684268951416 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.3499999%üƒ•_940395355 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3064900040626526 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6250206828117371 6 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.360289067029953 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.415340781211853 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10476675629615784 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3396151065826416 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.16697388887405396 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.31438490748405457 +jet/accuracy_3_of_6‚0.12280702590942383 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3472222089767456 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9865663051605225 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.13310185074806213 -jet/accuracy_4_of_5‚0.0026881720405071974 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.09719882905483246 particle/accuracy_1_of_4‚1 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.3125 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07106751203536987 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.46507686376571655 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.26916787028312683 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8181069493293762 particle/accuracy_1_of_5‚1 particle/accuracy_4_of_5‚1 jet/accuracy_5_of_6‚0 & Purity/2FRt3SRqqt1FBt/FRt_purity‚0 7validation_average_jet_accuracy‚0.37098145484924316 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3608774244785309 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.297682523727417 ,jet/accuracy_3_of_4‚0.048515237867832184 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9787962436676025 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.30128204822540283 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_2_of_3‚0.19713051617145538 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3464912474155426 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10247738659381866 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008386801928281784 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.13919679820537567 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.32380953431129456 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1388888955116272 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10508932173252106 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11190477758646011 ,jet/accuracy_3_of_3‚0.015445341356098652 particle/accuracy_1_of_2‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4761905074119568 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.16334150731563568 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3333333134651184 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.42202380299568176 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1432906538248062 *jet/accuracy_1_of_3‚0.7777447700500488 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8"Purity/1FRt0SRqqt1FBt/event_purity‚0.214690700173378 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3276596665382385 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.3044498860836029 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.31223076581954956 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3840048313140869 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9778092503547668 jet/accuracy_5_of_7‚0 jet/accuracy_6_of_7‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11210627853870392 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.19855967164039612 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666865348816 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4466840624809265 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 5 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.98939049243927 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11904069036245346 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3220689296722412 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4642857015132904 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08664652705192566 )jet/accuracy_1_of_5‚0.908699631690979 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.11172555387020111 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3286433517932892 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4469241201877594 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12374980747699738 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0007501875516027212 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1662410944700241 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.13851608335971832 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983593225479126 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3481253683567047 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.38348186016082764 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3473854660987854 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3328108489513397 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2581545412540436 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.10760369151830673 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.31790125370025635 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9893626570701599 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.331643670797348 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_3‚1 7"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.69047611951828 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9111111164093018 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 +particle/specificity‚0.9520881175994873 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.296875 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 % _timestamp‚1.7883920190941606e+09 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9920516014099121 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01346051599830389 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1127147302031517 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.35014206171035767 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.38170531392097473 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9959519505500793 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11887015402317047 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4135051667690277 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12769223749637604 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 *jet/accuracy_2_of_4‚0.2799793779850006 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4761905074119568 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.15824870765209198 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.040652625262737274 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 *jet/accuracy_1_of_2‚0.6499119400978088 *jet/accuracy_2_of_6‚0.2719298303127289 particle/accuracy_2_of_7‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 8"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.387152761220932 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3258417248725891 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4119017720222473 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.09305556118488312 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17773590981960297 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8847737312316895 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.699999988079071 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9981024861335754 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3434114158153534 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1066666692495346 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9946236610412598 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9901614785194397 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.3627195358276367 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.047122497111558914 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.10833333432674408 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3799999952316284 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.25833332538604736 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_5_of_7‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.47058823704719543 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8492063879966736 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3278224468231201 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03273938223719597 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.38575780391693115 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.33731091022491455 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3717948794364929 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_6‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04373015835881233 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.32728734612464905 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 particle/accuracy_3_of_7‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9893619418144226 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.38348186016082764 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8508745431900024 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.037442974746227264 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.11961270868778229 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.36182576417922974 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6643592119216919 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.30553528666496277 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38842904567718506 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.28883764147758484 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974292516708374 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.33552631735801697 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 epoch‚4 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.913293719291687 *jet/accuracy_1_of_1‚0.4091881215572357 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09885057061910629 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.32275131344795227 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16066157817840576 particle/accuracy_1_of_3‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.20190967619419098 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9873617887496948 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2870163023471832 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4717724919319153 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00452661607414484 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20347201824188232 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.018124794587492943 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.02500000037252903 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33728840947151184 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.33107393980026245 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8164142966270447 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34086254239082336 ,"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.375 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.3879578113555908 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.39504605531692505 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3190807104110718 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07256583869457245 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 8"Purity/1FRt0SRqqt*FBt/event_purity‚0.116067074239254 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_2_of_2‚0.0851321592926979 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9846647381782532 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9892777800559998 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.35817551612854004 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5222222208976746 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_7_of_7‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08024691790342331 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.002481389557942748 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_7_of_7‚0 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05562124401330948 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27391913533210754 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.28977569937705994 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16691015660762787 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ) Purity/*FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2385144978761673 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3171752095222473 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1Á åÁ] Íǧ ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1662410944700241 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.13851608335971832 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983593225479126 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3481253683567047 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.38348186016082764 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3473854660987854 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3328108489513397 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2581545412540436 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.10760369151830673 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.31790125370025635 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9893626570701599 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.331643670797348 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_3‚1 7"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.69047611951828 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9111111164093018 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 +particle/specificity‚0.9520881175994873 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.296875 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 % _timestamp‚1.7883920190941606e+09 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9920516014099121 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01346051599830389 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1127147302031517 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.35014206171035767 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.38170531392097473 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9959519505500793 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11887015402317047 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4135051667690277 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12769223749637604 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 *jet/accuracy_2_of_4‚0.2799793779850006 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4761905074119568 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.15824870765209198 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.040652625262737274 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 *jet/accuracy_1_of_2‚0.6499119400978088 *jet/accuracy_2_of_6‚0.2719298303127289 particle/accuracy_2_of_7‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 8"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.387152761220932 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3258417248725891 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4119017720222473 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.09305556118488312 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17773590981960297 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8847737312316895 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.699999988079071 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9981024861335754 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3434114158153534 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1066666692495346 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9946236610412598 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9901614785194397 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.3627195358276367 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.047122497111558914 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.10833333432674408 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3799999952316284 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.25833332538604736 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_5_of_7‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.47058823704719543 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8492063879966736 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3278224468231201 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03273938223719597 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.38575780391693115 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.33731091022491455 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3717948794364929 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_6‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04373015835881233 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.32728734612464905 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 particle/accuracy_3_of_7‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9893619418144226 :"Purity/0FRt1SRqqt0FBt/event>¡Üqs_purity‚0.38348186016082764 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8508745431900024 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.037442974746227264 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.11961270868778229 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.36182576417922974 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6643592119216919 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.30553528666496277 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38842904567718506 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.28883764147758484 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974292516708374 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.33552631735801697 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 epoch‚4 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.913293719291687 *jet/accuracy_1_of_1‚0.4091881215572357 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09885057061910629 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.32275131344795227 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16066157817840576 particle/accuracy_1_of_3‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.20190967619419098 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9873617887496948 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2870163023471832 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4717724919319153 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00452661607414484 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20347201824188232 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.018124794587492943 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.02500000037252903 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33728840947151184 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.33107393980026245 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8164142966270447 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34086254239082336 ,"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.375 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.3879578113555908 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.39504605531692505 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3190807104110718 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07256583869457245 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 8"Purity/1FRt0SRqqt*FBt/event_purity‚0.116067074239254 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_2_of_2‚0.0851321592926979 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9846647381782532 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9892777800559998 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.35817551612854004 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5222222208976746 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_7_of_7‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08024691790342331 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.002481389557942748 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_7_of_7‚0 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05562124401330948 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27391913533210754 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.28977569937705994 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.16691015660762787 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ) Purity/*FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2385144978761673 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3171752095222473 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.12745097279548645 jet/accuracy_5_of_5‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_1_of_7‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4028061032295227 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.33523640036582947 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26234322786331177 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 particle/accuracy_3_of_5‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1641712188720703 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2977273166179657 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.27232903242111206 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11951180547475815 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.3121572732925415 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 particle/accuracy_2_of_6‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 (particle/f_score‚0.38903990387916565 particle/accuracy_4_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3014635443687439 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.31127551198005676 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 _runtime‚2774.852318694 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.01558668166399002 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3241167366504669 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3110406994819641 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9943825602531433 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.15324676036834717 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4320240318775177 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9964770674705505 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2777777910232544 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1308000534772873 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.36947524547576904 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6447368264198303 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9019157886505127 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19838950037956238 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9987443089485168 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6622806787490845 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_4_of_6‚0.04385964944958687 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.007942832075059414 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_6‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13123808801174164 -jet/accuracy_4_of_4‚0.0021601382177323103 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 particle/accuracy_6_of_7‚1 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 _step‚244 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9953917264938354 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.08967042714357376 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986286759376526 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.33832475543022156 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.19949494302272797 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.32651278376579285 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3582138419151306 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 *loss/val_total_loss‚46.118554231121394 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07467762380838394 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016651051118969917 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3445245325565338 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08507245033979416 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31680843234062195 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.46666666865348816 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 *particle/sensitivity‚0.304080069065094 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.004597188904881477 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3592592775821686 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.02251780591905117 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4335184693336487 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2747695744037628 jet/accuracy_3_of_7‚0 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13207842409610748 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.32073697447776794 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.011290322057902813 particle/accuracy_4_of_4‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1388888955116272 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9912092685699463 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15533789992332458 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2668386399745941 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.33098095655441284 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2859530746936798 6 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.162566140294075 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.3499999940395355 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.15747852623462677 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 trainer/global_step‚3019 8"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.382296621799469 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0033064514864236116 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9884257912635803 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11594203114509583 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15768124163150787 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08507245033979416 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 *jet/accuracy_3_of_5‚0.0955095961689949 particle/accuracy_2_of_4‚1 particle/accuracy_4_of_7‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9926327466964722 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.14764520525932312 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_2_of_2‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3592684268951416 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.3499999940395355 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3064900040626526 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6250206828117371 6 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.360289067029953 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.415340781211853 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10476675629615784 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3396151065826416 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.16697388887405396 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.31438490748405457 +jet/accuracy_3_of_6‚0.12280702590942383 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3472222089767456 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9865663051605225 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.13310185074806213 -jet/accuracy_4_of_5‚0.0026881720405071974 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.09719882905483246 particle/accuracy_1_of_4‚1 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.3125 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07106751203536987 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.46507686376571655 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.26916787028312683 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8181069493293762 particle/accuracy_1_of_5‚1 particle/accuracy_4_of_5‚1 jet/accuracy_5_of_6‚0 & Purity/2FRt3SRqqt1FBt/FRt_purity‚0 7validation_average_jet_accuracy‚0.37098145484924316 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3608774244785309 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.297682523727417 ,jet/accuracy_3_of_4‚0.048515237867832184 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9787962436676025 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.30128204822540283 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_2_of_3‚0.19713051617145538 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3464912474155426 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10247738659381866 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008386801928281784 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.13919679820537567 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.32380953431129456 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1388888955116272 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10508932173252106 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11190477758646011 ,jet/accuracy_3_of_3‚0.015445341356098652 particle/accuracy_1_of_2‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4761905074119568 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.16334150731563568 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3333333134651184 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.42202380299568176 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1432906538248062 *jet/accuracy_1_of_3‚0.7777447700500488 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8"Purity/1FRt0SRqqt1FBt/event_purity‚0.214690700173378 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3276596665382385 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.3044498860836029 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.31223076581954956 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3840048313140869 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9778092503547668 jet/accuracy_5_of_7‚0 jet/accuracy_6_of_7‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11210627853870392 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.19855967164039612 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666865348816 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4466840624809265 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 5 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.98939049243927 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11904069036245346 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3220689296722412 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4642857015132904 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08664652705192566 )jet/accuracy_1_of_5‚0.908699631690979 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.11172555387020111 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3286433517932892 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4469241201877594 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12374980747699738 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0007501875516027212 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ("Purity/3FRt4SRqqt0FBt/event_purity‚0ôÝb r›Î‚(: ÓäâÔÙæ¯u gpu.0.gpu‚5.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚30.0 gpu.0.powerWatts‚88.683' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.736600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0·dîLÏ‚(:Á ÓäâÔœ‹‘x& disk.nvme0n1.out‚21949.00048828125 disk.rbd2.in‚0! disk.nvme0c0n1.in‚ 80.88671875 disk.rbd3.out‚ 1.44140625 disk.rbd1.in‚0 cpu‚25.395294424674073 proc.cpu.threads‚17 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚0.109375 network.sent‚ 5.985484e+06 network.recv‚955413 disk.sda.in‚0.109375 disk.rbd0.out‚0.078125( disk.nvme0c0n1.out‚21949.00048828125 disk.nvme0n1.in‚ 80.88671875 disk.rbd0.in‚0 disk.sda2.out‚0 disk.dm-1.out‚ 118.44921875 disk.rbd1.out‚ 1.595703125( proc.memory.availableMB‚ 1729.1953125! disk.rbd2.out‚315.94580078125 disk.sda.out‚ 118.44921875 disk.sda1.in‚0$ memory_percent‚89.44582939147949" proc.memory.rssMB‚ 2891.5859375* proc.memory.percent‚17.648839950561523) disk./.usagePercent‚70.62524213192589 disk.drbd1777.out‚0 disk.dm-0.in‚ 80.88671875# disk.dm-0.out‚21949.00048828125 disk.sda3.in‚0.109375 disk.sda3.out‚ 118.44921875# disk./.usageGB‚1052.36714553833ó¶aœÐ‚(:‘ âäâÔäÓu gpu.0.gpu‚48.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.08' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.216000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ü–Ü?Ñ‚(:´ âäâÔ”…ðw" disk.dm-0.out‚22099.8740234375! disk.rbd2.out‚323.35205078125 disk.sda1.out‚0 network.sent‚ 6.088045e+06 disk.rbd2.in‚0 disk.sda.in‚0.109375 disk.sda.out‚ 119.640625! disk.nvme0c0n1.in‚ 80.88671875' disk.nvme0c0n1.out‚22099.8740234375$ memory_percent‚89.45808410644531 proc.memory.rssMB‚ 2893.71875) disk./.usagePercent‚70.69972649998296 disk.dm-0.in‚ 80.88671875 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 119.640625 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.109375 disk.rbd1.out‚ 1.595703125 disk.nvme0n1.in‚ 80.88671875% disk.nvme0n1.out‚22099.8740234375 disk.sda1.in‚0% proc.memory.availableMB‚ 1727.1875 cpu‚25.396100866050247 disk.rbd1.in‚0% disk./.usageGB‚1053.4770164489746 disk.rbd3.out‚ 1.44140625 disk.sda3.out‚ 119.640625 disk.sda2.in‚0) proc.memory.percent‚17.66185760498047 proc.cpu.threads‚17 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.drbd1777.in‚0 network.recv‚960899ñ: Ÿ2Ò- _wandbruntime‚2801  _runtime‚2801-ÜÕ†jÓÁ & lr-AdamW/pg2‚0.0009943078154319052 trainer/global_step‚3049 % _timestamp‚1.78839204619L“4à]28012e+09 _runtime‚2801.948757825 _step‚245 & lr-AdamW/pg1‚0.0009943078154319052bÞË2ÌÔÆ % _timestamp‚1.7883920461928012e+09 _runtime‚2801.948757825 _step‚245 & lr-AdamW/pg1‚0.0009943078154319052 & lr-AdamW/pg2‚0.0009943078154319052 trainer/global_step‚3049õ ûÊ…2Õ- _wandbruntime‚2801  _runtime‚2801÷ï.ç'ÖáO JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚203051085661754 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2502846766650 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚52369976 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚207210356561408 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2503038763190 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚205553716548608 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2517070222848 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚65109607 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16373612 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚200548238895104 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚205553733587968 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2641603023360 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚52369974 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 117456983 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚65109607 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚208070590592512 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚208070590592512 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 117479583 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚208070590592512 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚208070590592512 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 117479583 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2503038763190 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚205553716548608 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2517070222848 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚65109607 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16373612 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚200548238895104 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚205553733587968 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2641603023360 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚52369974 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 117456983 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750öø&Í2Ø- _wandbruntime‚2802  _runtime‚28023cºò&ÙìM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚205618772624896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚52386496 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚207275857154048 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2503640649598 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2517672434688 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚52386496 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚205618755585536 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚205618755585536 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚203115687965386 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚48489362 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚208136428020224 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚32111234 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16378128 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚203115478929278 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚65107881 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16378625 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚200611855319040 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚205618772624896 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2642688980992 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2642455257088 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚200611838279680 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚65107881 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚208136641251840 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚3049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚52386494 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2517672434688 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 117494375 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚247 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚65130505 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 117494375 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 117517001 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚207275857154048 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2503640649598 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2517672434688 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚52386496 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚205618755585536 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚205618755585536 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚203115687965386 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚48489362 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚208136428020224 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚32111234 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16378128 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚203115478929278 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚65107881 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16378625 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚200611855319040 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚205618772624896 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2642688980992 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2642455257088 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚200611838279680 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚65107881 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚208136641251840 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚3049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚52386494 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2517672434688 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 117494375 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚247 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚65130505 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 117494375 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 117517001 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚211362489806336 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚211362489806336 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚3099 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2683976605696 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚210486631377408 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2683747062784 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚210486606473728 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚206270985245862 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚211362703024640 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254446592 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2542760082598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚495 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚53195976 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2557017590272 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚66133961 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚213170607983104 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚49231776 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16628873 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚66156559 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚66133961 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚208805685434368 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚16628378 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2556821411328 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚32602903 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2542568086058 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚203728208123904 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚53195974 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 % _timestamp‚1.7883920878115408e+09 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚208805668395008 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2557017590272 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 119352535 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚49231278 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚53195976 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚250 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 119329935 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚208805668395008Y%góÂéçO F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚211362703024640 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚53195974 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚206270776209962 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚211362489806336 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚3099 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2683976605696 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚210486631377408 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2683747062784 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚210486606473728 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚206270985245862 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚211362703024640 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254446592 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2542760082598 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚495 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚53195976 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2557017590272 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚66133961 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚213170607983104 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚49231776 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16628873 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚66156559 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚66133961 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚208805685434368 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚16628378 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2556821411328 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_$ƒRç+start/active.all.peak‚27581 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚32602903 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2542568086058 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚203728208123904 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚53195974 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 % _timestamp‚1.7883920878115408e+09 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚208805668395008 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2557017590272 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 119352535 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚49231278 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚53195976 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _step‚250 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 119329935 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚208805668395008 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚66156559 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚213170353536512 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚32602900 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 229542912 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚211362489806336 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚66177457 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16634097 % _timestamp‚1.7883920885558844e+09 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2684827290112 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2543361969006 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2557619802112 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _runtime‚2844.312354289 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚66154833 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚208870724471296 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚203791807508480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚211428540465664 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚53212494 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚66177457 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚206335378513594 trainer/global_step‚3099 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 119389953 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚49247034 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚203791824547840 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚53212496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 119367327 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚208870707431936 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚49246511 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16633577 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚213236984163840 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚210552131970048 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚66154833 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2557815994368 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2543553965754 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚53212496 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0Ä0îÇ÷&ìñM J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚66154833 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2557815994368 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2543553965754 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚53212496 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 119367327 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚66177457 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16634097 % _timestamp‚1.7883920885558844e+09 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2684827290112 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2543361969006 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2557619802112 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _runtime‚2844.312354289 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚66154833 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚208870724471296 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚203791807508480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚211428540465664 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚53212494 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚66177457 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚206335378513594 trainer/global_step‚3099 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 119389953 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚49247034 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚203791824547840 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚53212496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 119367327 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚208870707431936 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚49246511 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16633577 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚213236984163840 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚210552131970048 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824ûýé{2í- _wandbruntime‚2844  _runtime‚2844i¨Š=XîÒ 0loss/FRt3/assignment_loss‚3.1307195030063033 2loss/SRqqt3/assignment_loss‚1.5092749698087573 /loss/FRt1/detection_loss‚2.5288752506497088 epoch‚5 $ _timestamp‚1.788392088558068e+09 /loss/FRt2/detection_loss‚1.7233271392659075 0loss/SRqqt4/detection_loss‚2.381780313528501 -loss/FBt3/detection_loss‚2.49952411254247 -loss/FBt4/detection_loss‚4.73367977142334 _step‚252 2loss/SRqqt2/assignment_loss‚1.6384903404232063 1loss/SRqqt2/detection_loss‚1.0429227093995457 .loss/FRt4/detection_loss‚2.800291621685028 /loss/FBt1/detection_loss‚3.9660835936665535 0loss/FRt1/assignment_loss‚3.2393981218338013 1loss/FBt2/assignment_loss‚0.15025878851155505 0loss/FBt3/assignment_loss‚0.1291561814257875 /loss/FRt3/detection_loss‚2.3930728507329184 _runtime‚2844.313877108 /loss/FRt4/assignment_loss‚4.061182301694696 0loss/SRqqt1/assignment_loss‚1.70059376589402 1loss/SRqqt4/assignment_loss‚2.035085485302485 1loss/FBt4/assignment_loss‚0.23037813762202858 /loss/FBt2/detection_loss‚0.5574573233314505 %loss/total_loss‚47.87654259418443 trainer/global_step‚3099 0loss/FRt2/assignment_loss‚1.8306394873313534 2loss/FBt1/assignment_loss‚0.033694697078317404 1loss/SRqqt1/detection_loss‚1.7551208472362272 1loss/SRqqt3/detection_loss‚1.8055352807904665¾æìt]ï× 1loss/SRqqt4/assignment_loss‚2.035085485302485 1loss/FBt4/assignment_loss‚0.23037813762202858 /loss/FBt2/detection_loss‚0.5574573233314505 %loss/total_loss‚47.87654259418443 trainer/global_step‚3099 0loss/FRt2/assignment_loss‚1.8306394873313534 2loss/FBt1/assignment_loss‚0.033694697078317404 1loss/SRqqt1/detection_loss‚1.7551208472362272 1loss/SRqqt3/detection_loss‚1.8055352807904665 0loss/FRt3/assignment_loss‚3.1307195030063033 2loss/SRqqt3/assignment_loss‚1.5092749698087573 /loss/FRt1/detection_loss‚2.5288752506497088 epoch‚5 $ _timestamp‚1.788392088558068e+09 /loss/FRt2/detection_loss‚1.7233271392659075 0loss/SRqqt4/detection_loss‚2.381780313528501 -loss/FBt3/detection_loss‚2.49952411254247 -loss/FBt4/detection_loss‚4.73367977142334 _step‚252 2loss/SRqqt2/assignment_loss‚1.6384903404232063 1loss/SRqqt2/detection_loss‚1.0429227093995457 .loss/FRt4/detection_loss‚2.800291621685028 /loss/FBt1/detection_loss‚3.9660835936665535 0loss/FRt1/assignment_loss‚3.2393981218338013 1loss/FBt2/assignment_loss‚0.15025878851155505 0loss/FBt3/assignment_loss‚0.1291561814257875 /loss/FRt3/detection_loss‚2.3930728507329184 _runtime‚2844.313877108 /loss/FRt4/assignment_loss‚4.061182301694696 0loss/SRqqt1/assignment_loss‚1.70059376589402üà§ôð‚(:’ žåâÔ½ÁÈu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.709' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.341800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Úòð?ñ‚(:´ žåâÔŠÉäx disk.rbd1.out‚ 1.595703125 disk.nvme0n1.in‚ 85.15234375 disk.sda3.in‚0.109375 cpu‚25.401410456627218 disk.dm-1.in‚0.109375 disk.dm-1.out‚ 123.578125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 1.50390625 disk.dm-0.in‚ 85.15234375$ disk./.usageGB‚1057.576000213623 disk.sda1.out‚0' disk.nvme0c0n1.out‚27320.0029296875 disk.sda2.in‚0 disk.rbd2.out‚334.1884765625% disk.nvme0n1.out‚27320.0029296875 disk.sda.out‚ 123.578125" disk.dm-0.out‚27320.0029296875 proc.memory.rssMB‚ 2893.71875 network.sent‚ 6.19282e+06) disk./.usagePercent‚70.97481273970497 disk.sda.in‚0.109375 disk.rbd3.in‚0 disk.sda3.out‚ 123.578125$ memory_percent‚89.46185111999512) proc.memory.percent‚17.66185760498047 proc.cpu.threads‚17 network.recv‚979413 disk.rbd2.in‚0 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 85.15234375( proc.memory.availableMB‚ 1726.5703125 disk.rbd1.in‚0 disk.rbd0.out‚0.078125v¾ò‚(:’ ­åâÔÊõØu gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.734' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.146800000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0KZ&/ó‚(:¤ ­åâÔÍÏ•x network.recv‚984744 disk.sda2.in‚0$ disk./.usageGB‚1062.964153289795 disk.drbd1777.out‚0 disk.dm-1.in‚0.109375 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.rbd3.in‚0) proc.memory.availableMB‚ 1726.82421875$ disk.nvme0c0n1.out‚ 31788.7109375 disk.sda2.out‚0 disk.drbd1777.in‚0 disk.dm-0.out‚ 31788.7109375 disk.rbd2.in‚0) proc.memory.percent‚17.66185760498047) disk./.usagePercent‚71.33641621360842 disk.rbd3.out‚ 1.98828125 disk.dm-0.in‚ 103.984375 disk.dm-1.out‚124.9375 disk.rbd2.out‚334.2666015625 disk.sda.in‚0.109375$ memory_percent‚89.46030139923096 network.sent‚ 6.279641e+06 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd0.out‚0.078125 disk.nvme0c0n1.in‚ 103.984375 disk.sda3.out‚124.9375 disk.rbd1.in‚0 disk.rbd1.out‚ 1.595703125 disk.nvme0n1.in‚ 103.984375" disk.nvme0n1.out‚ 31788.7109375 proc.memory.rssMB‚ 2893.71875 disk.sda3.in‚0.109375 disk.sda.out‚124.9375 cpu‚25.402592845766357‘Ða!’ô‚(:‡ ¼åâÔªÔÐu gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.668' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.9336 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0+:/1õ‚(:¦ ¼åâÔëêx disk.rbd3.out‚ 1.98828125 disk.rbd1.out‚ 1.595703125 disk.sda.in‚0.109375 disk.sda1.in‚0 network.recv‚989438) disk./.usagePercent‚71.52996226162217 disk.dm-0.in‚ 103.984375 disk.sda.out‚125.1875 disk.nvme0c0n1.in‚ 103.984375% disk.nvme0c0n1.out‚34899.37890625 proc.cpu.threads‚17 disk.dm-1.in‚0.109375 disk.dm-1.out‚125.1875$ disk./.usageGB‚1065.848129272461 disk.sda3.in‚0.109375 proc.memory.rssMB‚ 2893.71875 disk.sda2.out‚0 disk.rbd2.in‚0 disk.rbd2.out‚335.4306640625 disk.nvme0n1.in‚ 103.984375# disk.nvme0n1.out‚34899.37890625 disk.sda1.out‚0) proc.memory.percent‚17.66185760498047 disk.sda3.out‚125.1875 disk.rbd1.in‚0 disk.rbd0.in‚0 network.sent‚ 6.285861e+06 disk.dm-0.out‚34899.37890625 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ memory_percent‚89.46032524108887 cpu‚25.403824476451426 disk.sda2.in‚0 disk.rbd3.in‚0( proc.memory.availableMB‚ 1726.8203125ûžö2ö- _wandbruntime‚2885  _runtime‚2885àŽÆ÷À & lr-AdamW/pg1‚0.0009450657821131689 & lr-AdamW/pg2‚0.0009450657821131689 trainer/global_step‚3149 % _timestamp‚1.7883921298404584e+09 _runtime‚ 2885.59641608 _step‚253õ?êþËøÅ & lr-AdamW/pg1‚0.0009450657821131689 & lr-AdamW/pg2‚0.0009450657821131689 trainer/global_step‚3149 % _timestamp‚1.7883921298404584e+09 _runtime‚ 2885.59641608 _step‚253ý2{:2ù- _wandbruntime‚2885  _runtime‚2885G:ö6õúâO 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚206908194392064 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚33104603 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273320960 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚216489051247104 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2596964957696 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚527 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚213762906193408 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚54021974 \HDeviceStatsMfIdó"onitor.on_train_batch_start/allocated_bytes.large_pool.freed‚212057620241408 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚54021974 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚209490466758170 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2582289405466 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16887091 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚206908177352704 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2596768778752 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883921298414538e+09 _runtime‚2885.597648793 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚212057637280768 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _step‚254 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚214654389020160 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚33104600 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚216488777926144 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 121225487 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2596964957696 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2726145053696 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 121202887 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚67203511 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2582481402006 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚49991164 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚530 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚214654602238464 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚54021976 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2725896636416 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚212057637280768 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 trainer/global_step‚3149 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2596768778752 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚16886564 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚67180913 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚67203511 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560)‚¹ží'ûçO TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚16886564 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚67180913 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚67203511 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚206908194392064 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚33104603 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273320960 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚216489051247104 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2596964957696 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚527 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚213762906193408 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚54021974 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚212057620241408 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚54021974 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚209490466758170 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2582289405466 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚16887091 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚206908177352704 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2596768778752 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883921298414538e+09 _runtime‚2885.597648793 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚212057637280768 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 _step‚254 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚214654389020160 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚33104600 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚216488777926144 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 121225487 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2596964957696 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2726145053696 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 121202887 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚67203511 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2582481402006 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚49991164 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚530 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚214654602238464 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚54021976 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2725896636416 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚212057637280768 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 trainer/global_step‚3149 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2596768778752 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2þò·pÓ2ü- _wandbruntime‚2886  _runtime‚2886s¨Ö'ñ&ýëM S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚214720226447872 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚67224409 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚212122659278336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚213828431689728 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2726984904192 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚54038496 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚514 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2597567169536 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚ 2886.33996771 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2597567169536 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚209554860025694 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2726740694528 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚50006379 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚50006896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 trainer/global_step‚3149 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚206971793776640 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚216555416593920 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚255 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2597763361792 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚214720439679488 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚212122676317696 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 121262905 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚67224409 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2583083288414 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚54038494 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚67201785 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚216555147480576 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 121240279 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚54038496 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 121262905 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2597763361792 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚213828406786048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚212122676317696 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 121262905 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚67224409 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2583083288414 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚54038494 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚67201785 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚216555147480576 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 121240279 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚54038496 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 121262905 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2597763361792 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚213828406786048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚ 2886.33996771 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2597567169536 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚209554860025694 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2726740694528 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚50006379 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚50006896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 trainer/global_step‚3149 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚206971793776640 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚216555416593920 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚255 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750ÿ†Àé™2ÿ- _wandbruntime‚2886  _runtime‚2886kE×6\€Ö 0loss/FRt2/assignment_loss‚1.8499784580374186 1loss/SRqqt1/assignment_loss‚1.757623046535898 2loss/SRqqt3/assignment_loss‚1.8518602732413758 /loss/FRt2/detection_loss‚1.8293758837786396 .loss/FBt4/detection_loss‚2.744066961109638 1loss/FBt3/assignment_loss‚0.19320980267811036 % _timestamp‚1.7883921305851862e+09 0loss/FRt4/assignment_loss‚4.4162532687187195 2loss/SRqqt2/assignment_loss‚1.4313094484482387 2loss/SRqqt4/assignment_loss‚1.7140383270748876 /loss/FRt1/detection_loss‚2.4250724711351923 1loss/SRqqt3/detection_loss‚1.9744866097656388 _step‚256 0loss/FBt4/assignment_loss‚0.2265142246421116 /loss/FRt3/detection_loss‚2.5857605315744876 1loss/SRqqt4/detection_loss‚2.5769152254389045 epoch‚5 %loss/total_loss‚48.03711965793225 _runtime‚2886.340971411 1loss/SRqqt1/detection_loss‚1.8257569912317637 1loss/SRqqt2/detection_loss‚1.1397970201539211 .loss/FBt1/detection_loss‚4.298629144827525 0loss/FBt1/assignment_loss‚0.1284818797527502 1loss/FBt2/assignment_loss‚0.17018650634593235 /loss/FRt4/detection_loss‚2.4258387974566884 .loss/FBt3/detection_loss‚2.452327287859387 /loss/FRt1/assignment_loss‚3.430062774817149 0loss/FRt3/assignment_loss‚3.9648289531469345 /loss/FBt2/detection_loss‚0.6247457701609367 trainer/global_step‚3149oêaÛ 0loss/FBt4/assignment_loss‚0.2265142246421116 /loss/FRt3/detection_loss‚2.5857605315744876 1loss/SRqqt4/detection_loss‚2.5769152254389045 epoch‚5 %loss/total_loss‚48.03711965793225 _runtime‚2886.340971411 1loss/SRqqt1/detection_loss‚1.8257569912317637 1loss/SRqqt2/detection_loss‚1.1397970201539211 .loss/FBt1/detection_loss‚4.298629144827525 0loss/FBt1/assignment_loss‚0.1284818797527502 1loss/FBt2/assignment_loss‚0.17018650634593235 /loss/FRt4/detection_loss‚2.4258387974566884 .loss/FBt3/detection_loss‚2.452327287859387 /loss/FRt1/assignment_loss‚3.430062774817149 0loss/FRt3/assignment_loss‚3.9648289531469345 /loss/FBt2/detection_loss‚0.6247457701609367 trainer/global_step‚3149 0loss/FRt2/assignment_loss‚1.8499784580374186 1loss/SRqqt1/assignment_loss‚1.757623046535898 2loss/SRqqt3/assignment_loss‚1.8518602732413758 /loss/FRt2/detection_loss‚1.8293758837786396 .loss/FBt4/detection_loss‚2.744066961109638 1loss/FBt3/assignment_loss‚0.19320980267811036 % _timestamp‚1.7883921305851862e+09 0loss/FRt4/assignment_loss‚4.4162532687187195 2loss/SRqqt2/assignment_loss‚1.4313094484482387 2loss/SRqqt4/assignment_loss‚1.7140383270748876 /loss/FRt1/detection_loss‚2.4250724711351923 1loss/SRqqt3/detection_loss‚1.9744866097656388 _step‚256€²§Ô‚‚(:’ ËåâÔè–Ñu gpu.0.gpu‚51.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.847' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.169400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ZÕ|µFƒ‚(:» ËåâÔÜÚ¯x disk.sda2.out‚0( proc.memory.availableMB‚ 1583.8203125% disk./.usageGB‚1065.9206733703613 disk.sda3.out‚ 125.953125 disk.sda1.in‚0 disk.rbd3.out‚ 2.05126953125# disk.dm-0.out‚36718.37939453125$ memory_percent‚90.33312797546387 disk.rbd1.out‚ 1.595703125 disk.sda1.out‚0 disk.rbd0.in‚0 disk.dm-0.in‚ 236.1640625 proc.cpu.threads‚17 network.recv‚993316 disk.sda.in‚0.109375! disk.nvme0c0n1.in‚ 236.1640625 proc.memory.rssMB‚ 2893.71875 disk.sda3.in‚0.109375 disk.dm-1.out‚ 125.953125 disk.sda.out‚ 125.953125 disk.rbd0.out‚0.078125 disk.drbd1777.out‚0) proc.memory.percent‚17.66185760498047 cpu‚25.40506383164621 disk.rbd1.in‚0( disk.nvme0c0n1.out‚36718.37939453125 disk.sda2.in‚0 disk.rbd3.in‚0 network.sent‚ 6.29127e+06 disk.rbd2.in‚0 disk.drbd1777.in‚0 disk.dm-1.in‚0.109375! disk.rbd2.out‚335.47412109375 disk.nvme0n1.in‚ 236.1640625& disk.nvme0n1.out‚36718.37939453125) disk./.usagePercent‚71.53483075690082Bì@{„‚(:’ ÚåâÔþ’¶w gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.288' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.257600000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¥“~H…‚(:½ ÚåâÔ±¿Ûy network.sent‚ 6.378169e+06 disk.sda.out‚ 126.328125 disk.sda1.in‚0 disk.rbd0.in‚0 proc.memory.rssMB‚ 2893.71875 disk.nvme0n1.in‚ 256.1796875 disk.sda.in‚0.109375( disk.nvme0c0n1.out‚37689.92431640625 disk.sda3.in‚0.109375& disk.nvme0n1.out‚37689.92431640625# disk.dm-0.out‚37689.92431640625$ memory_percent‚90.33491611480713 disk.rbd3.out‚ 2.05126953125 disk.rbd1.out‚ 1.595703125 disk.dm-1.out‚ 126.328125) disk./.usagePercent‚71.53536350857958 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚335.48583984375$ disk./.usageGB‚1065.928611755371 disk.sda1.out‚0 disk.rbd0.out‚0.078125! disk.nvme0c0n1.in‚ 256.1796875 disk.sda2.out‚0 disk.dm-0.in‚ 256.1796875) proc.memory.percent‚17.66185760498047 disk.dm-1.in‚0.109375 proc.cpu.threads‚17 network.recv‚998557 disk.sda2.in‚0 disk.sda3.out‚ 126.328125) proc.memory.availableMB‚ 1583.52734375 cpu‚25.406176162155987 disk.rbd1.in‚0hT‘†‚(:† éåâÔ°ƒœu gpu.0.gpu‚57.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚94.221' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.8442 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Î%¦ŠA‡‚(:¶ éåâÔúñ•y! disk.rbd2.out‚336.56005859375 disk.sda.in‚ 0.11328125) proc.memory.availableMB‚ 1583.51953125) proc.memory.percent‚17.66185760498047 proc.cpu.threads‚17 disk.sda3.in‚ 0.11328125 network.sent‚ 6.383568e+06! disk.nvme0c0n1.in‚ 259.2265625 disk.rbd1.out‚ 1.595703125 network.recv‚ 1.001957e+06 disk.rbd1.in‚0 disk.sda1.out‚0 disk.sda3.out‚ 143.3359375! disk.nvme0c0n1.out‚ 39407.1875 disk.sda2.out‚0 disk.dm-0.in‚ 259.2265625 disk.dm-0.out‚ 39407.1875 disk.drbd1777.out‚0 proc.memory.rssMB‚ 2893.71875 disk.dm-1.in‚ 0.11328125 disk.nvme0n1.out‚ 39407.1875 disk.sda1.in‚0 disk.rbd2.in‚0% disk./.usageGB‚1067.1914978027344 disk.sda2.in‚0 disk.nvme0n1.in‚ 259.2265625 disk.sda.out‚ 143.3359375 disk.rbd0.in‚0 cpu‚25.407558192218648 disk.rbd0.out‚0.078125( disk./.usagePercent‚71.6201168508501 disk.rbd3.in‚0 disk.dm-1.out‚ 143.3359375 disk.drbd1777.in‚0$ memory_percent‚90.33496379852295 disk.rbd3.out‚ 2.1142578125ö¼ü2ˆ- _wandbruntime‚2927  _runtime‚292780êƉÀ trainer/global_step‚3199 $ _timestamp‚1.788392172125309e+09 _runtime‚2927.881285037 _step‚257 & lr-AdamW/pg1‚0.0009203270497177259 & lr-AdamW/pg2‚0.0009203270497177259÷ÍËŠÅ & lr-AdamW/pg1‚0.0009203270497177259 & lr-AdamW/pg2‚0.0009203270497177259 trainer/global_step‚3199 $ _timestamp‚1.788392172125309e+09 _runtime‚2927.881285037 _step‚257‚f%¶2‹- _wandbruntime‚2927  _runtime‚2927;mç'ŒáO \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2622202721414 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚217946288233984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚496 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚212710157306378 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2768107840000 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17145715 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2636912325120 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2636716146176 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚217039181009408 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2622010724874 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚50752018 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚212710366342278 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚68250463 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚215309589127168 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2636912325120 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚54847974 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚210088163620864 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚217039156105728 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 123098439 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 $ _timestamp‚1.788392172126365e+09 _runtime‚2927.882551704 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2636716146176 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚50751522 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 123075839 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚493 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚54847974 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚33606300 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚219807023917056 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912†cþ æO PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚68250463 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚215309589127168 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2636912325120 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚54847974 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚210088163620864 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚217039156105728 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 123098439 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 $ _timestamp‚1.788392172126365e+09 _runtime‚2927.882551704 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2636716146176 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚50751522 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 123075839 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚493 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚54847974 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚33606300 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚219807023917056 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2622202721414 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚217946288233984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚496 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚212710157306378 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2768107840000 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17145715 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2636912325120 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2636716146176 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚217039181009408 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2622010724874 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚50752018 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚212710366342278 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824‚®’®…2Ž- _wandbruntime‚2928  _runtime‚2928¨Uò&ìM ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚210151745966080 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚218012338893312 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2637514536960 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚218012338893312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚50767306 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚54864496 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚68271361 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚217104706505728 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚3199 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚215374628164096 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚68248737 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚215374611124736 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 123135857 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚508 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚219873381806080 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚50766798 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚33616337 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2768700204032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2637710729216 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2768944413696 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚212774759610010 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883921729036334e+09 _runtime‚2928.660199479 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚217104681602048 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚218012125661696 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚215374611124736 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2622996604570 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 123135857 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚215374628164096 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚68271361 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚212774550573902 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚210151763005440 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2637710729216~$÷&ñM S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚218012125661696 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚215374611124736 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2622996604570 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 123135857 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚215374628164096 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚68271361 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚212774550573902 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚210151763005440 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2637710729216 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚210151745966080 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚218012338893312 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2637514536960 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚218012338893312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚50767306 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚54864496 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚68271361 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚217104706505728 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚3199 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚215374628164096 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚68248737 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚215374611124736 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 123135857 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚508 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚219873381806080 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚50766798 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚33616337 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2768700204032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2637710729216 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2768944413696 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚212774759610010 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883921729036334e+09 _runtime‚2928.660199479 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚217104681602048 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824ƒ6✦2‘- _wandbruntime‚2928  _runtime‚2928Ü6=ê\’Ö 2loss/SRqqt1/assignment_loss‚1.6774574930134873 2loss/SRqqt2/assignment_loss‚1.4573161225713842 .loss/FRt2/detection_loss‚1.827063088460515 1loss/SRqqt3/detection_loss‚1.9999049465323604 trainer/global_step‚3199 _runtime‚2928.661268032 1loss/SRqqt3/assignment_loss‚1.584445037034362 2loss/FBt4/assignment_loss‚0.052232001714097955 0loss/SRqqt4/detection_loss‚2.420056571563085 .loss/FBt3/detection_loss‚2.601075733701388 &loss/total_loss‚44.888827903134136 /loss/FRt3/assignment_loss‚3.374225462807549 /loss/FRt1/detection_loss‚2.4269232925066015 0loss/FRt1/assignment_loss‚3.3999088160770454 .loss/FRt4/detection_loss‚2.597826668194362 /loss/FBt1/detection_loss‚2.6183405085043474 /loss/FBt2/detection_loss‚0.5530476780342204 /loss/FRt4/assignment_loss‚4.062609929697854 1loss/FBt2/assignment_loss‚0.15721842810663209 2loss/SRqqt4/assignment_loss‚1.8122868788165925 /loss/FRt3/detection_loss‚2.4249820712539885 0loss/FRt2/assignment_loss‚1.6969311231126387 1loss/SRqqt1/detection_loss‚1.8051940699895987 /loss/FBt4/detection_loss‚2.9906892627477646 $ _timestamp‚1.788392172905505e+09 1loss/FBt1/assignment_loss‚0.11699670954575678 1loss/FBt3/assignment_loss‚0.25501582441664067 0loss/SRqqt2/detection_loss‚0.977080184731867 epoch‚5 _step‚2608Œ‡Va“Û 0loss/SRqqt2/detection_loss‚0.977080184731867 epoch‚5 _step‚260 2loss/SRqqt1/assignment_loss‚1.6774574930134873 2loss/SRqqt2/assignment_loss‚1.4573161225713842 .loss/FRt2/detection_loss‚1.827063088460515 1loss/SRqqt3/detection_loss‚1.9999049465323604 trainer/global_step‚3199 _runtime‚2928.661268032 1loss/SRqqt3/assignment_loss‚1.584445037034362 2loss/FBt4/assignment_loss‚0.052232001714097955 0loss/SRqqt4/detection_loss‚2.420056571563085 .loss/FBt3/detection_loss‚2.601075733701388 &loss/total_loss‚44.888827903134136 /loss/FRt3/assignment_loss‚3.374225462807549 /loss/FRt1/detection_loss‚2.4269232925066015 0loss/FRt1/assignment_loss‚3.3999088160770454 .loss/FRt4/detection_loss‚2.597826668194362 /loss/FBt1/detection_loss‚2.6183405085043474 /loss/FBt2/detection_loss‚0.5530476780342204 /loss/FRt4/assignment_loss‚4.062609929697854 1loss/FBt2/assignment_loss‚0.15721842810663209 2loss/SRqqt4/assignment_loss‚1.8122868788165925 /loss/FRt3/detection_loss‚2.4249820712539885 0loss/FRt2/assignment_loss‚1.6969311231126387 1loss/SRqqt1/detection_loss‚1.8051940699895987 /loss/FBt4/detection_loss‚2.9906892627477646 $ _timestamp‚1.788392172905505e+09 1loss/FBt1/assignment_loss‚0.11699670954575678 1loss/FBt3/assignment_loss‚0.25501582441664067„aä_‘”‚(:† øåâÔû¡v gpu.0.gpu‚22.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚96.887' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.3774 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ðLVÊ[•‚(:Ð øåâÔ„ûûx network.recv‚ 1.006585e+06 disk.drbd1777.out‚0 disk.rbd1.in‚0 cpu‚25.40875615579137 disk.dm-1.out‚ 144.74609375 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0 disk.dm-0.in‚ 260.08203125! disk.rbd2.out‚336.56005859375 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.in‚0# disk.dm-0.out‚39512.36962890625 disk.rbd2.in‚0) proc.memory.percent‚17.66185760498047 network.sent‚ 6.389789e+06 disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-1.in‚ 0.11328125 disk.sda.in‚ 0.11328125" disk.nvme0c0n1.in‚ 260.08203125 disk.rbd1.out‚ 1.595703125) disk./.usagePercent‚71.61976151239257 disk.nvme0n1.in‚ 260.08203125 disk.rbd3.out‚ 2.1142578125 disk.sda3.in‚ 0.11328125 disk.sda3.out‚ 144.74609375( proc.memory.availableMB‚ 1583.4921875 proc.cpu.threads‚17% disk./.usageGB‚1067.1862030029297& disk.nvme0n1.out‚39512.36962890625 disk.sda.out‚ 144.74609375( disk.nvme0c0n1.out‚39512.36962890625$ memory_percent‚90.33513069152832 proc.memory.rssMB‚ 2893.71875ÁÆ ’–‚(:‡ ‡æâÔòÊÂu gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.493' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.8986 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)f&R—‚(:Ç ‡æâÔÕÚõx disk.drbd1777.out‚0 disk.rbd1.out‚ 1.595703125! disk.rbd2.out‚336.66552734375 proc.memory.rssMB‚ 2893.71875 network.recv‚ 1.01324e+06 disk.sda1.in‚0 disk.rbd3.in‚0) proc.memory.percent‚17.66185760498047 disk.sda1.out‚0" disk.nvme0c0n1.in‚ 260.31640625 disk.sda2.out‚0 disk.dm-1.in‚ 0.11328125$ disk./.usageGB‚1067.187198638916 disk.rbd3.out‚ 2.1142578125$ disk.nvme0n1.out‚39520.990234375$ memory_percent‚90.33668041229248' disk./.usagePercent‚71.619828330359 disk.rbd0.out‚0.078125! disk.dm-0.out‚39520.990234375 disk.rbd1.in‚0 disk.sda.out‚ 155.71484375 cpu‚25.40977639341154& disk.nvme0c0n1.out‚39520.990234375 disk.sda.in‚ 0.11328125 disk.sda2.in‚0 disk.sda3.in‚ 0.11328125 disk.sda3.out‚ 155.71484375) proc.memory.availableMB‚ 1583.23828125 proc.cpu.threads‚17 disk.drbd1777.in‚0 disk.dm-0.in‚ 260.31640625 disk.dm-1.out‚ 155.71484375 disk.rbd2.in‚0 disk.nvme0n1.in‚ 260.31640625 network.sent‚ 6.477867e+06 disk.rbd0.in‚0$'íU‘˜‚(:† –æâÔôð£u gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.095' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.019 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¹,bS™‚(:È –æâÔøÑçw disk.sda.in‚ 0.1171875 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚336.66552734375 proc.memory.rssMB‚ 2893.71875) proc.memory.percent‚17.66185760498047 disk.dm-1.in‚ 0.1171875) disk./.usagePercent‚71.61983242647955 disk.sda.out‚ 168.27734375 disk.sda1.out‚0# memory_percent‚90.3336763381958 disk.rbd0.out‚0.078125 disk.sda2.out‚0 disk.rbd3.in‚0 disk.dm-1.out‚ 168.27734375) proc.memory.availableMB‚ 1583.73046875 cpu‚25.410967399017597 disk.rbd1.out‚ 1.595703125 disk.rbd1.in‚0% disk./.usageGB‚1067.1872596740723 disk.nvme0n1.in‚ 260.31640625$ disk.nvme0n1.out‚39619.291015625& disk.nvme0c0n1.out‚39619.291015625 disk.sda3.in‚ 0.1171875 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd3.out‚ 2.1142578125 disk.dm-0.in‚ 260.31640625 disk.sda3.out‚ 168.27734375 network.recv‚ 1.016953e+06 proc.cpu.threads‚17 network.sent‚ 6.483216e+06 disk.drbd1777.out‚0" disk.nvme0c0n1.in‚ 260.31640625! disk.dm-0.out‚39619.291015625¼Õ¶.2š- _wandbruntime‚2970  _runtime‚2970Ñ øÇ›Á & lr-AdamW/pg1‚0.0008955373604172776 & lr-AdamW/pg2‚0.0008955373604172776 trainer/global_step‚3249 % _timestamp‚1.7883922146244884e+09 _runtime‚2970.380900576 _step‚261œõ°¬ÌœÆ _step‚261 & lr-AdamW/pg1‚0.0008955373604172776 & lr-AdamW/pg2‚0.0008955373604172776 trainer/global_step‚3249 % _timestamp‚1.7883922146244884e+09 _runtime‚2970.380900576…GÜ42- _wandbruntime‚2970  _runtime‚2970Õãð žâO VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚221238400666112 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚34108000 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚215930056890486 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 124971391 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚221238187447808 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2809802874880 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2810047097856 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚221238187447808 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚522 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.aMp øll.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚223125502923264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2661732044282 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚3249 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚69297415 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚218561540973568 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚218561540973568 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚220315455825408 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚221238400666112 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2676663513600 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚69274817 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2676859692544 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883922146252348e+09 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2676663513600 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚218561523934208 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚220315430921728 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17405942 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚69274817 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2676859692544 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883922146252348e+09 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2676663513600 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚218561523934208 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚220315430921728 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17405942 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚221238187447808 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2809802874880 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2810047097856 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚221238187447808 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚522 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚223125502923264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2661732044282 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 trainer/global_step‚3249 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚69297415 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚218561540973568 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚218561540973568 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚220315455825408 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚221238400666112 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2676663513600 J6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 124986183 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883922153884823e+09 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚213331732234240 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚213331715194880 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚69295689 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚51529616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚34118037 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚220380956418048 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚17411579 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 _runtime‚2971.145143145 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2810878834176 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 _step‚263 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚55690496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚55690494 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚218626562971136 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚55690496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚69295689 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚3249 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 125008809 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚69318313 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚51529089 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2662525927230 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 125008809 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚218626562971136 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚55690496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚69295689 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚3249 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 125008809 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚69318313 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚51529089 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2662525927230 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 125008809 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 124986183 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883922153884823e+09 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚213331732234240 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚213331715194880 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚69295689 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚51529616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚34118037 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚220380956418048 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚17411579 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 _runtime‚2971.145143145 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2810878834176 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 _step‚263 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚55690496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚55690494 PDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚224530086661632 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚221813492819968 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚216448085039104 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17667011 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2851673049088 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17666510 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚34609700 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2716610881024 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚224530086661632 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚221813475780608 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚224530299879936 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚56499974 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 126844343 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚3299 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚70321769 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2716610881024 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚224530086661632 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚221813475780608 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚224530299879936 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚56499974 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 126844343 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚3299 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚70321769 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚224530086661632 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚221813492819968 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚216448085039104 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17667011 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2851673049088 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17666510 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚34609700 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚216511701463040 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚216511684423680 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 126881761 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚70342641 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚226510008539648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚52291426 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2717605464064 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚70365265 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2702439243386 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚221878514817536 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚56516496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚56516496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚56516494 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 126881761 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚224596137320960 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚221878531856896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2717409271808 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚34619737 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚224596137320960 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚52291933 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 126859135 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚224595924089344 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚221878514817536 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚34619734 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2852752401920 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚70342641 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0(÷&²ñM J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚56516494 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2852512386560 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚216511701463040 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚216511684423680 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 126881761 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚70342641 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚226510008539648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚52291426 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2717605464064 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚70365265 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2702439243386 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚221878514817536 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚56516496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚56516496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚56516494 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 126881761 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚224596137320960 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚221878531856896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2717409271808 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚34619737 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚224596137320960 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚52291933 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 126859135 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚224595924089344 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚221878514817536 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚34619734 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2852752401920 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚70342641 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚219214140706426 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚535 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚57325976 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚57325974 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883922994854968e+09 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚35111403 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚229761670225408 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2741366679638 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚225065444666368 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2893923408896 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚227821985875456 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17925128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚222369228951002 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚227822199093760 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚225065427627008 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17925663 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 128717295 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚229761928866304 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocateÖ_vd‚ 509607936 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚225065427627008 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚227822199093760 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚71391319 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚538 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _step‚270 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚57325976 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚57325974 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2756558248448 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚71368721 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚53036528 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚219628054267904 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚535 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚57325976 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚57325974 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883922994854968e+09 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚35111403 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚229761670225408 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2741366679638 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚225065444666368 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2893923408896 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚227821985875456 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚17925128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚222369228951002 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚227822199093760 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚225065427627008 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚17925663 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 128717295 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚229761928866304 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚225065427627008 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚227822199093760 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚71391319 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚538 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _step‚270 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚57325976 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚57325974 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2756558248448 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚71368721 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚53036528 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚219628054267904 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚222433831254634 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2742160562794 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚229828303128064 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚227887823303168 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2757356639232 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚35121434 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚57342494 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2741968566046 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚57342496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2757552831488 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2757552831488 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚57342496 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚71412217 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 128754713 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚229828042403328 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2894536353280 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚227888036534784 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚225130483703296 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 128754713 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 128732087 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚53052103 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚226933530953728 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚271 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 128732087 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚71389593 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚17930666 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚219691670691840 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2757356639232 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚227888036534784 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚519 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚17930147 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚222433622218526 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚219691653652480 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚57342494 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚225130466663936 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚226933506050048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2894772174336 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚53051581 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507~å´²ÄðM H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 128754713 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚229828042403328 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2894536353280 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚227888036534784 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚225130483703296 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 128754713 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 128732087 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚53052103 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚226933530953728 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚271 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 128732087 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚71389593 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚17930666 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚219691670691840 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2757356639232 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚227888036534784 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚519 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚17930147 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚222433622218526 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚219691653652480 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚57342494 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚225130466663936 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚226933506050048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2894772174336 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚53051581 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚225130466663936 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚71389593 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚35121437 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚222433831254634 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2742160562794 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚229828303128064 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚227887823303168 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2757356639232 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚35121434 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚57342494 =7DeviceStatsMonitD}[Dor.on_train_batch_end/num_alloc_retries‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2741968566046 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚57342496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2757552831488 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2757552831488 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚57342496 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚71412217 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0òq/;2Å- _wandbruntime‚3056  _runtime‚30561Àð[ÆÕ 0loss/FBt1/assignment_loss‚0.2382735970084988 1loss/FBt3/assignment_loss‚0.37619934176010167 /loss/FBt1/detection_loss‚2.8420418245451793 trainer/global_step‚3349 /loss/FRt3/assignment_loss‚3.413656988641718 2loss/SRqqt3/assignment_loss‚1.7813524416262152 &loss/total_loss‚45.051482165412025 1loss/SRqqt1/detection_loss‚1.7292549454227641 .loss/FBt3/detection_loss‚2.447343360293995 epoch‚5 % _timestamp‚1.7883923002518253e+09 /loss/FRt2/detection_loss‚1.6091898374050577 /loss/FBt2/detection_loss‚0.6114024587185873 _runtime‚3056.007676243 /loss/FRt1/assignment_loss‚3.450436593383871 2loss/SRqqt1/assignment_loss‚2.0113650336516677 0loss/FBt4/assignment_loss‚0.2576376434248232 1loss/SRqqt2/detection_loss‚1.0529237484340281 _step‚272 2loss/SRqqt2/assignment_loss‚1.0433430010589826 1loss/FBt2/assignment_loss‚0.16768317330846264 .loss/FRt4/detection_loss‚2.604215686236109 1loss/SRqqt4/detection_loss‚2.4064799073864434 /loss/FBt4/detection_loss‚2.3398772150278093 .loss/FRt3/detection_loss‚2.497101211613351 0loss/FRt2/assignment_loss‚1.8553492627411245 2loss/SRqqt4/assignment_loss‚1.8223961231243961 .loss/FRt1/detection_loss‚2.334942822975497 1loss/SRqqt3/detection_loss‚1.9104443746383744 /loss/FRt4/assignment_loss‚4.248571572984968™»Á,`ÇÚ /loss/FRt1/assignment_loss‚3.450436593383871 2loss/SRqqt1/assignment_loss‚2.0113650336516677 0loss/FBt4/assignment_loss‚0.2576376434248232 1loss/SRqqt2/detection_loss‚1.0529237484340281 _step‚272 2loss/SRqqt2/assignment_loss‚1.0433430010589826 1loss/FBt2/assignment_loss‚0.16768317330846264 .loss/FRt4/detection_loss‚2.604215686236109 1loss/SRqqt4/detection_loss‚2.4064799073864434 /loss/FBt4/detection_loss‚2.3398772150278093 .loss/FRt3/detection_loss‚2.497101211613351 0loss/FRt2/assignment_loss‚1.8553492627411245 2loss/SRqqt4/assignment_loss‚1.8223961231243961 .loss/FRt1/detection_loss‚2.334942822975497 1loss/SRqqt3/detection_loss‚1.9104443746383744 /loss/FRt4/assignment_loss‚4.248571572984968 0loss/FBt1/assignment_loss‚0.2382735970084988 1loss/FBt3/assignment_loss‚0.37619934176010167 /loss/FBt1/detection_loss‚2.8420418245451793 trainer/global_step‚3349 /loss/FRt3/assignment_loss‚3.413656988641718 2loss/SRqqt3/assignment_loss‚1.7813524416262152 &loss/total_loss‚45.051482165412025 1loss/SRqqt1/detection_loss‚1.7292549454227641 .loss/FBt3/detection_loss‚2.447343360293995 epoch‚5 % _timestamp‚1.7883923002518253e+09 /loss/FRt2/detection_loss‚1.6091898374050577 /loss/FBt2/detection_loss‚0.6114024587185873 _runtime‚3056.007676243S[‚Í’È‚(:‡ ðæâÔàüÃu gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.528' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.9056 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0áˆæEÉ‚(:º ðæâÔ±Â×x network.sent‚ 6.681017e+06! disk.dm-0.out‚40022.080078125 disk.dm-1.in‚ 0.1171875 disk.sda.in‚ 0.1171875 disk.sda.out‚ 210.40625 network.recv‚ 1.044921e+06 disk.sda3.in‚ 0.1171875 disk.drbd1777.in‚0& disk.nvme0c0n1.out‚40022.080078125( disk./.usagePercent‚71.5308096465605% disk./.usageGB‚1065.8607559204102 disk.rbd3.out‚ 2.1142578125 disk.dm-0.in‚ 260.3203125 disk.rbd0.out‚0.078125 disk.rbd3.in‚0 disk.dm-1.out‚ 210.40625 disk.drbd1777.out‚0" memory_percent‚90.336012840271 proc.memory.rssMB‚ 2893.71875 disk.rbd2.in‚0 disk.nvme0n1.in‚ 260.3203125 proc.cpu.threads‚17 disk.rbd1.out‚ 1.595703125! disk.rbd2.out‚341.19677734375 disk.sda1.in‚0) proc.memory.percent‚17.66185760498047$ disk.nvme0n1.out‚40022.080078125 disk.sda2.in‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 1583.34765625 disk.sda3.out‚ 210.40625 disk.rbd1.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 260.3203125 cpu‚25.4178303902318358HÊ Ê‚(:’ ÿæâÔœ¦Úu gpu.0.gpu‚37.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚157.568' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.513600000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0^kÓTË‚(:É ÿæâÔᇳx# memory_percent‚90.3362512588501 proc.memory.rssMB‚ 2893.71875! disk.nvme0c0n1.in‚ 260.3203125 disk.drbd1777.out‚0 disk.dm-0.in‚ 260.3203125 disk.dm-1.out‚ 223.47265625 disk.rbd1.in‚0( disk.nvme0c0n1.out‚40023.36083984375 disk.sda3.out‚ 223.47265625 disk.sda.out‚ 223.47265625 disk.sda1.in‚0 disk.drbd1777.in‚0 network.sent‚ 6.768982e+06% disk./.usageGB‚1065.8607559204102 disk.sda2.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚341.21240234375 disk.nvme0n1.in‚ 260.3203125 disk.sda.in‚ 0.1171875 disk.sda1.out‚0 disk.rbd3.in‚0& disk.nvme0n1.out‚40023.36083984375 disk.rbd0.in‚0) proc.memory.percent‚17.66185760498047 network.recv‚ 1.050911e+06( disk./.usagePercent‚71.5308096465605 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875) proc.memory.availableMB‚ 1583.30859375 cpu‚25.41900895636248 disk.sda2.out‚0 disk.rbd3.out‚ 2.1142578125# disk.dm-0.out‚40023.36083984375 proc.cpu.threads‚17 disk.rbd1.out‚ 1.595703125 disk.rbd0.out‚0.078125[¡P’Ì‚(:‡ ŽçâÔô™Ûu gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.668' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.9336 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,\ýâQÍ‚(:Æ ŽçâÔ‚ŸÓw& disk.nvme0n1.out‚40121.58740234375 disk.sda1.out‚0% disk./.usageGB‚1065.8607559204102( disk.nvme0c0n1.out‚40121.58740234375 disk.sda2.out‚0 disk.rbd0.out‚0.078125) proc.memory.percent‚17.66185760498047 cpu‚25.41994950446534 network.sent‚ 6.774273e+06 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd0.in‚0 disk.sda3.out‚ 224.5078125 disk.rbd2.in‚0 disk.rbd3.out‚ 2.1142578125 disk.dm-0.in‚ 260.3203125 disk.sda3.in‚ 0.1171875$ memory_percent‚90.33627510070801 proc.cpu.threads‚17( disk./.usagePercent‚71.5308096465605 disk.drbd1777.in‚0 disk.sda.out‚ 224.5078125 disk.sda1.in‚0 network.recv‚ 1.054742e+06# disk.dm-0.out‚40121.58740234375 disk.rbd1.out‚ 1.595703125! disk.rbd2.out‚341.22802734375 disk.sda.in‚ 0.1171875( proc.memory.availableMB‚ 1583.3046875 proc.memory.rssMB‚ 2893.71875 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 260.3203125 disk.rbd3.in‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 224.5078125 disk.nvme0n1.in‚ 260.3203125‰"Y2Î- _wandbruntime‚3097  _runtime‚3097 â;ÆÏÀ _step‚273 & lr-AdamW/pg1‚0.0008210696369311805 & lr-AdamW/pg2‚0.0008210696369311805 trainer/global_step‚3399 % _timestamp‚1.7883923421432984e+09 _runtime‚ 3097.89928083Æðñ2ËÐÅ trainer/global_step‚3399 % _timestamp‚1.7883923421432984e+09 _runtime‚ 3097.89928083 _step‚273 & lr-AdamW/pg1‚0.0008210696369311805 & lr-AdamW/pg2‚0.0008210696369311805‘R½z2Ñ- _wandbruntime‚3097  _runtime‚3097èRÅRè'ÒâO JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚231114098307584 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚233079924711424 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚230144255369728 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚540 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚230144280273408 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚231113885089280 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 130590247 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚222808040536064 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚58151974 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚53797732 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚228317396512768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚58151976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚72438271 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚72415673 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚228317379473408 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2796505615872 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚18184629 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚274 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2796701794816 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 trainer/global_step‚3399 _runtime‚3097.904028544 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚58151974 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚225588919499210 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚231114098307584 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7883923421443722e+09 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚72415673 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2935915661824 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2935669341696 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚228317396512768 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚222808023496704 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚53797192 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 130567647 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2781087999046 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2780896002506 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2796701794816 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚58151976 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚233080195935232 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚35613103 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚231113885089280ñ}í´í'ÓçO E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2796505615872 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚228317379473408 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚222808040536064 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚58151974 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚53797732 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚228317396512768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚58151976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚72438271 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚72415673 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚228317379473408 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2796505615872 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚18184629 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚274 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2796701794816 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 trainer/global_step‚3399 _runtime‚3097.904028544 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚58151974 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚225588919499210 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚231114098307584 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7883923421443722e+09 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚72415673 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2935915661824 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2935669341696 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚228317396512768 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚222808023496704 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚53797192 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 130567647 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2781087999046 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2780896002506 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2796701794816 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚58151976 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚233080195935232 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚35613103 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚231113885089280 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚231114098307584 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚233079924711424 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚230144255369728 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚540 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚230144280273408 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚231113885089280 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 130590247 JDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 130627665 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚72436545 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚53812302 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚228382418510336 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚58168496 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚222871622881280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚529 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚225653521802842 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚225653312766734 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚58168496 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2797500198912 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18189168 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚72436545 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _step‚275 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 130605039 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚58168494 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚3098.658043699 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 130627665 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚231179935748608 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚72459169 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚222871639920640 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚230209780866048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 130605039 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18189697 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2936521106432 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚3399HXÙ8÷&ÖñM VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚35623137 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 % _timestamp‚1.7883923429014301e+09 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚72436545 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _step‚275 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 130605039 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚58168494 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚3098.658043699 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 130627665 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚231179935748608 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚72459169 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚222871639920640 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚230209780866048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 130605039 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18189697 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2936521106432 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚3399 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚228382435549696 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚231179722516992 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚35623134 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚233146301972480 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚231179935748608 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚233146560600064 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚228382418510336 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚230209805769728 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 130627665 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚72436545 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚53812302 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚228382418510336 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚58168496 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚222871622881280 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚529 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚225653521802842 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚225653312766734 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚58168496 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2797500198912 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18189168 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚234405784303104 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚225988009764864 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2820809318454 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2836452983296 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚54559858 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚73485223 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚231569331319808 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 277515264 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚236398283388928 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚225987992725504 _step‚278 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚231569331319808 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2836649162240 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚228808819083318 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚58977974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2977753203200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 % _timestamp‚1.7883923844483361e+09 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚234405997521408 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2836452983296 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚3140.204545309 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2836649162240 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚231569348359168 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚54559324 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚18444524 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚234405997521408 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚231569348359168 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚36114800 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚73462625 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚234405784303104 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚225988009764864 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2820809318454 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2836452983296 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚54559858 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚73485223 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚231569331319808 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 277515264 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚236398283388928 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚225987992725504 _step‚278 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚231569331319808 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2836649162240 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚228808819083318 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚58977974 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2977753203200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 % _timestamp‚1.7883923844483361e+09 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚234405997521408 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2836452983296 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚3140.204545309 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2836649162240 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚231569348359168 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚54559324 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚18444524 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304–õ<Óf2æ- _wandbruntime‚3140  _runtime‚3140‹Š¡ò&çìM _step‚279 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚36124837 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2837251374080 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚234471621730816 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 132477991 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚73483497 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚231634387396096 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚54575116 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚58994494 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚231634370356736 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 132500617 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 132477991 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚236464652269568 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚233486055682048 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚54574587 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚228873003314942 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚226051592110080 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2821603201610 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚73506121 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚228873212351050 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 132500617 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2837447566336 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚529 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚36124834 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚73506121 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚236464912994304 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚231634387396096 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2837447566336 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883923852094789e+09}Òq²%èñM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚36124837 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2837251374080 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚234471621730816 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 132477991 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚73483497 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚231634387396096 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚54575116 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚58994494 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚231634370356736 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 132500617 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 132477991 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚236464652269568 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚233486055682048 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚54574587 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚228873003314942 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚226051592110080 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2821603201610 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚73506121 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚228873212351050 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 132500617 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2837447566336 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚529 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚36124834 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚73506121 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚236464912994304 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚231634387396096 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2837447566336 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883923852094789e+09 _step‚279 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚36616503 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚234821300205568 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚239716739473408 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3019909568000 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚237697683516928 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚234821283166208 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚59803976 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚74532175 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚18705350 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚55321318 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚232028509631526 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚36616503 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚234821300205568 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚239716739473408 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3019909568000 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚237697683516928 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚234821283166208 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚59803976 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚74532175 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚18705350 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚55321318 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚232028509631526 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0šÏ=§g2ø- _wandbruntime‚3183  _runtime‚3183ÝRs—ùëM aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3020754878464 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚74553073 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚237763734176256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚59820496 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚59820494 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 134350943 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18710124 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 134373569 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 trainer/global_step‚3499 % _timestamp‚1.7883924279455497e+09 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18710628 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚283 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚74530449 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚74553073 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚74530449 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2861132524270 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_e²åZnd/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚234886339242496 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚232092693863150 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚229231578378240 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚234886322203136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2877198741504 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2877394933760 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚55337165 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2877198741504 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚59820494 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚232092902899258 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚36626534 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚236762330498048 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 134373569 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚237763734176256 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3020506474496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚237763520944640 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚55336658 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚234886339242496 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚239782836972544 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760º¹à°ö&úðM WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚59820494 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 134350943 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18710124 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 134373569 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 trainer/global_step‚3499 % _timestamp‚1.7883924279455497e+09 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18710628 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚283 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚74530449 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚74553073 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚74530449 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2861132524270 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚234886339242496 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚232092693863150 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚229231578378240 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚234886322203136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2877198741504 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2877394933760 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚55337165 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2877198741504 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚59820494 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚232092902899258 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚36626534 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚236762330498048 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 134373569 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚237763734176256 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3020506474496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚237763520944640 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚55336658 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚234886339242496 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚239782836972544 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3020754878464 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚74553073 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚237763734176256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚59820496 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624›»J>-2û- _wandbruntime‚3183  _runtime‚3183¥Rkà\üÖ /loss/FRt4/assignment_loss‚4.163462920391813 2loss/SRqqt3/assignment_loss‚1.5515074426164994 1loss/FBt3/assignment_loss‚0.06776840168672874 1loss/SRqqt1/detection_loss‚1.5718125627707626 1loss/SRqqt3/detection_loss‚2.0229733689473224 /loss/FBt1/detection_loss‚2.4725639931857586 1loss/FBt1/assignment_loss‚0.37212509484658085 .loss/FRt3/detection_loss‚2.442163701568331 epoch‚5 0loss/FRt2/assignment_loss‚1.9532288714896802 /loss/FRt3/assignment_loss‚3.135841938711348 %loss/total_loss‚45.27595349330027 1loss/SRqqt2/assignment_loss‚1.258729882000221 2loss/SRqqt4/assignment_loss‚2.0158514612407052 1loss/FBt2/assignment_loss‚0.06740717271455686 1loss/FBt4/assignment_loss‚0.34911378423243555 .loss/FBt4/detection_loss‚2.436066766579946 trainer/global_step‚3499 _runtime‚3183.703384552 _step‚284 /loss/FRt2/detection_loss‚1.9109968192182307 2loss/SRqqt1/assignment_loss‚1.9247734975385464 1loss/SRqqt2/detection_loss‚0.9380846562208953 0loss/SRqqt4/detection_loss‚2.584872012993075 /loss/FBt2/detection_loss‚0.5583331772783662 /loss/FBt3/detection_loss‚2.3259237934561336 % _timestamp‚1.7883924279475598e+09 0loss/FRt1/assignment_loss‚3.3875105074710317 .loss/FRt1/detection_loss‚2.579635148247083 /loss/FRt4/detection_loss‚3.1852065178942173#¦naýÛ 0loss/FRt1/assignment_loss‚3.3875105074710317 .loss/FRt1/detection_loss‚2.579635148247083 /loss/FRt4/detection_loss‚3.1852065178942173 /loss/FRt4/assignment_loss‚4.163462920391813 2loss/SRqqt3/assignment_loss‚1.5515074426164994 1loss/FBt3/assignment_loss‚0.06776840168672874 1loss/SRqqt1/detection_loss‚1.5718125627707626 1loss/SRqqt3/detection_loss‚2.0229733689473224 /loss/FBt1/detection_loss‚2.4725639931857586 1loss/FBt1/assignment_loss‚0.37212509484658085 .loss/FRt3/detection_loss‚2.442163701568331 epoch‚5 0loss/FRt2/assignment_loss‚1.9532288714896802 /loss/FRt3/assignment_loss‚3.135841938711348 %loss/total_loss‚45.27595349330027 1loss/SRqqt2/assignment_loss‚1.258729882000221 2loss/SRqqt4/assignment_loss‚2.0158514612407052 1loss/FBt2/assignment_loss‚0.06740717271455686 1loss/FBt4/assignment_loss‚0.34911378423243555 .loss/FBt4/detection_loss‚2.436066766579946 trainer/global_step‚3499 _runtime‚3183.703384552 _step‚284 /loss/FRt2/detection_loss‚1.9109968192182307 2loss/SRqqt1/assignment_loss‚1.9247734975385464 1loss/SRqqt2/detection_loss‚0.9380846562208953 0loss/SRqqt4/detection_loss‚2.584872012993075 /loss/FBt2/detection_loss‚0.5583331772783662 /loss/FBt3/detection_loss‚2.3259237934561336 % _timestamp‚1.7883924279475598e+09œ…£X;‘þ‚(:† ÷çâÔÓÜÌu gpu.0.gpu‚28.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚119.045' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.809 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0bLPÿ‚(:Å ÷çâÔÓûèx disk.sda3.in‚ 0.1171875$ disk.nvme0n1.out‚40666.560546875 disk.sda.in‚ 0.1171875 disk.rbd3.in‚0) proc.memory.percent‚17.66185760498047 disk.dm-1.in‚ 0.1171875! disk.rbd2.out‚342.45068359375 disk.nvme0n1.in‚ 260.3359375 disk.sda.out‚ 289.22265625 disk.sda1.out‚0 disk.sda2.in‚0 disk.dm-0.in‚ 260.3359375 network.sent‚ 6.977929e+06) disk./.usagePercent‚71.53109688701392 disk.drbd1777.in‚0 proc.cpu.threads‚17 network.recv‚ 1.087145e+06! disk.nvme0c0n1.in‚ 260.3359375$ memory_percent‚90.33958911895752 cpu‚25.42725193671907 disk.rbd1.out‚ 1.595703125 disk.rbd2.in‚0 disk.sda3.out‚ 289.22265625 disk.rbd1.in‚0& disk.nvme0c0n1.out‚40666.560546875 disk.rbd3.out‚ 2.1142578125% disk./.usageGB‚1065.8650360107422 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0 proc.memory.rssMB‚ 2893.71875 disk.dm-1.out‚ 289.22265625 disk.rbd0.out‚0.078125 disk.sda2.out‚0! disk.dm-0.out‚40666.560546875) proc.memory.availableMB‚ 1582.76171875<¡x œ€‚(:‘ †èâÔÒøÇu gpu.0.gpu‚35.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚92.037' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.407400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚05èGjT‚(:É †èâÔËÉñw disk.dm-1.out‚ 289.7109375 proc.memory.rssMB‚ 2893.71875 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.sda.out‚ 289.7109375 disk.rbd0.out‚0.078125% disk./.usageGB‚1065.8650360107422! disk.nvme0c0n1.in‚ 260.3359375( disk.nvme0c0n1.out‚40666.95654296875 disk.dm-1.in‚ 0.1171875 disk.sda.in‚ 0.1171875 disk.sda1.in‚0) proc.memory.availableMB‚ 1582.75390625 network.recv‚ 1.092545e+06 disk.drbd1777.out‚0# disk.dm-0.out‚40666.95654296875 disk.rbd1.in‚0 disk.rbd3.out‚ 2.1142578125 disk.sda3.out‚ 289.7109375 disk.nvme0n1.in‚ 260.3359375$ memory_percent‚90.33963680267334 network.sent‚ 7.064654e+06 disk.rbd3.in‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 1.595703125) proc.memory.percent‚17.66185760498047) disk./.usagePercent‚71.53109688701392 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 cpu‚25.428350459338926! disk.rbd2.out‚342.51318359375& disk.nvme0n1.out‚40666.95654296875 disk.sda2.out‚0 disk.dm-0.in‚ 260.3359375 ËÌ‘‚‚(:† •èâÔóŒw gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚153.445' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.689 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÝáÒÎMƒ‚(: •èâÔÇõ¤y network.recv‚ 1.096182e+06% disk./.usageGB‚1065.8652801513672 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.1142578125 disk.dm-1.out‚ 301.3515625 proc.cpu.threads‚17 disk.nvme0n1.in‚ 260.3359375& disk.nvme0c0n1.out‚40766.248046875 disk.sda1.in‚0 disk.sda2.out‚0! disk.dm-0.out‚40766.248046875 disk.sda3.in‚ 0.1171875$ memory_percent‚90.33818244934082 proc.memory.rssMB‚ 2893.71875) proc.memory.percent‚17.66185760498047) disk./.usagePercent‚71.53111327149611 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd1.out‚ 1.681640625( proc.memory.availableMB‚ 1582.9921875 cpu‚25.429251649944728 disk.sda1.out‚0 disk.sda3.out‚ 301.3515625 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.sda.out‚ 301.3515625 network.sent‚ 7.070022e+06 disk.dm-0.in‚ 260.3359375 disk.rbd1.in‚0$ disk.nvme0n1.out‚40766.248046875 disk.rbd0.out‚0.078125 disk.drbd1777.in‚0! disk.rbd2.out‚343.60302734375³À2„- _wandbruntime‚3225  _runtime‚3225‘Å•Ç…Á & lr-AdamW/pg1‚0.0007468889993433411 & lr-AdamW/pg2‚0.0007468889993433411 trainer/global_step‚3549 % _timestamp‚1.7883924693619692e+09 _runtime‚3225.119003495 _step‚285€F}ãÌ†Æ & lr-AdamW/pg1‚0.0007468889993433411 & lr-AdamW/pg2‚0.0007468889993433411 trainer/global_step‚3549 % _timestamp‚1.7883924693619692e+09 _runtime‚3225.119003495 _step‚285Çv‚Š2‡- _wandbruntime‚3225  _runtime‚3225›¨Ÿ«"ˆâO =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚235247991143834 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚75556529 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2916543897088 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚60629976 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚60629974 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚238073252051968 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 % _timestamp‚1.7883924693627326e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 136209103 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚37118200 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚239973104721408 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚240989582730752 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3061515109376 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚531 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚243034876636672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚238073235012608 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚239973079817728 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚235248200179734 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2900251957270 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚232347931183104 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600|s_í'‰çO YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚238073235012608 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚239973079817728 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚235248200179734 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2900251957270 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚232347931183104 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚235247991143834 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚75556529 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2916543897088 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚60629976 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚60629974 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚238073252051968 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 % _timestamp‚1.7883924693627326e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 136209103 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚37118200 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚239973104721408 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚240989582730752 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3061515109376 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚531 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚243034876636672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0žE‚Îô2Š- _wandbruntime‚3225  _runtime‚3225’»Hò&‹ìM C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚75600025 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3062353486336 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2900853843678 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚232411547607040 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚37128237 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2917342301184 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚240038605314048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 136223895 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚240038630217728 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883924701257522e+09 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚241055633390080 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 136246521 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚243100958800384 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚241055633390080 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚241055420158464 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚75577401 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚232411530567680 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18969336 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3062601890304 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚238138291088896 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 136223895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2917342301184 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 _runtime‚3225.882330015 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 136246521 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚56098088 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 trainer/global_step‚3549 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚60646496 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚75600025 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚243101232108032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚56097570 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚37128234 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2901045840426 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚60646494 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚235312593447466 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18969851 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚287 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2917146108928 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507Ï û÷&ŒñM REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚18969336 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3062601890304 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚238138291088896 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 136223895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2917342301184 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 _runtime‚3225.882330015 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 136246521 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚56098088 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 trainer/global_step‚3549 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚60646496 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚75600025 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚243101232108032 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚56097570 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚37128234 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2901045840426 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚60646494 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚235312593447466 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚18969851 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚287 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2917146108928 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚75600025 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3062353486336 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2900853843678 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚232411547607040 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚37128237 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2917342301184 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚240038605314048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 136223895 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚240038630217728 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 % _timestamp‚1.7883924701257522e+09 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚241055633390080 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 136246521 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚243100958800384 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚241055633390080 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚241055420158464 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚75577401 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚232411530567680 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750Ÿ¾C¤î2- _wandbruntime‚3225  _runtime‚3225J€«^XŽÖ 0loss/SRqqt4/detection_loss‚2.238663722487057 1loss/SRqqt4/assignment_loss‚2.019863293100806 2loss/FBt1/assignment_loss‚0.048893094930978265 /loss/FBt2/detection_loss‚0.4977883032813508 %loss/total_loss‚45.56979564428836 1loss/FBt3/assignment_loss‚0.21991264598754617 1loss/FBt4/assignment_loss‚0.12615855957604558 /loss/FRt1/detection_loss‚2.2115028949067144 0loss/FRt1/assignment_loss‚3.4164892405566603 1loss/SRqqt1/assignment_loss‚2.005300605793794 trainer/global_step‚3549 _step‚288 2loss/SRqqt3/assignment_loss‚1.8211053947217009 1loss/SRqqt2/detection_loss‚1.0427603386626898 epoch‚5 % _timestamp‚1.7883924701281135e+09 1loss/FBt2/assignment_loss‚0.15835361693656255 /loss/FRt3/detection_loss‚2.7057508546859026 .loss/FBt3/detection_loss‚2.907036956618814 1loss/SRqqt1/detection_loss‚1.7259144350886344 /loss/SRqqt3/detection_loss‚2.19214234760638 /loss/FBt1/detection_loss‚2.4817967005074024 /loss/FBt4/detection_loss‚2.8062149584293365 _runtime‚3225.883971508 /loss/FRt2/detection_loss‚1.79534ðØó&14432704449 /loss/FRt4/detection_loss‚2.2885764241218567 0loss/FRt2/assignment_loss‚1.8237607166880654 /loss/FRt3/assignment_loss‚3.706277371942997 /loss/FRt4/assignment_loss‚4.151225471798377 2loss/SRqqt2/assignment_loss‚1.1789662525882476yTÑaÛ 0loss/FRt1/assignment_loss‚3.4164892405566603 1loss/SRqqt1/assignment_loss‚2.005300605793794 trainer/global_step‚3549 _step‚288 2loss/SRqqt3/assignment_loss‚1.8211053947217009 1loss/SRqqt2/detection_loss‚1.0427603386626898 epoch‚5 % _timestamp‚1.7883924701281135e+09 1loss/FBt2/assignment_loss‚0.15835361693656255 /loss/FRt3/detection_loss‚2.7057508546859026 .loss/FBt3/detection_loss‚2.907036956618814 1loss/SRqqt1/detection_loss‚1.7259144350886344 /loss/SRqqt3/detection_loss‚2.19214234760638 /loss/FBt1/detection_loss‚2.4817967005074024 /loss/FBt4/detection_loss‚2.8062149584293365 _runtime‚3225.883971508 /loss/FRt2/detection_loss‚1.7953414432704449 /loss/FRt4/detection_loss‚2.2885764241218567 0loss/FRt2/assignment_loss‚1.8237607166880654 /loss/FRt3/assignment_loss‚3.706277371942997 /loss/FRt4/assignment_loss‚4.151225471798377 2loss/SRqqt2/assignment_loss‚1.1789662525882476 0loss/SRqqt4/detection_loss‚2.238663722487057 1loss/SRqqt4/assignment_loss‚2.019863293100806 2loss/FBt1/assignment_loss‚0.048893094930978265 /loss/FBt2/detection_loss‚0.4977883032813508 %loss/total_loss‚45.56979564428836 1loss/FBt3/assignment_loss‚0.21991264598754617 1loss/FBt4/assignment_loss‚0.12615855957604558 /loss/FRt1/detection_loss‚2.2115028949067144 w‹M‚(:’ ¤èâÔÍÇÓu gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.402' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.480399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¿è¥£J‘‚(:¿ ¤èâÔŠûÁx disk.sda2.in‚0 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0.078125 disk.rbd2.in‚0 disk.dm-0.in‚ 260.3359375 disk.sda3.out‚ 312.86328125 disk.dm-1.out‚ 312.86328125% proc.memory.availableMB‚ 1582.6875 disk.sda3.in‚ 0.1171875 proc.memory.rssMB‚ 2893.71875 proc.cpu.threads‚17( disk./.usagePercent‚71.5311288879557$ disk.nvme0n1.out‚40766.642578125 disk.rbd3.out‚ 2.1142578125 disk.dm-1.in‚ 0.1171875! disk.rbd2.out‚343.66552734375$ memory_percent‚90.34004211425781 disk.rbd1.out‚ 1.744140625 disk.drbd1777.out‚0 network.sent‚ 7.076304e+06 network.recv‚ 1.10116e+06 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 260.3359375% disk./.usageGB‚1065.8655128479004 disk.sda.out‚ 312.86328125 disk.sda1.out‚0 disk.sda2.out‚0! disk.dm-0.out‚40766.642578125 disk.nvme0n1.in‚ 260.3359375 disk.sda.in‚ 0.1171875& disk.nvme0c0n1.out‚40766.642578125 disk.rbd3.in‚0) proc.memory.percent‚17.66185760498047 cpu‚25.43025389100654 disk.rbd1.in‚0†{á’’‚(:‡ ³èâÔÄ¢‘u gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.542' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.7084 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢ãD2R“‚(:Ç ³èâÔÁÌ­x% disk.nvme0n1.out‚40865.0595703125 disk.sda.out‚ 325.83984375' disk.nvme0c0n1.out‚40865.0595703125 disk.dm-0.in‚ 260.3359375 disk.sda3.in‚ 0.1171875 proc.cpu.threads‚17 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.rbd3.in‚0! disk.rbd2.out‚343.67724609375 proc.memory.rssMB‚ 2893.71875% disk./.usageGB‚1065.8655319213867 disk.nvme0n1.in‚ 260.3359375$ memory_percent‚90.34006595611572 cpu‚25.4311633696443) disk./.usagePercent‚71.53113016799337 disk.rbd3.out‚ 2.1142578125" disk.dm-0.out‚40865.0595703125 disk.sda3.out‚ 325.83984375) proc.memory.percent‚17.66185760498047 disk.rbd0.in‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 network.recv‚ 1.107574e+06! disk.nvme0c0n1.in‚ 260.3359375 disk.dm-1.out‚ 325.83984375 disk.rbd1.out‚ 1.744140625) proc.memory.availableMB‚ 1582.68359375 disk.sda.in‚ 0.1171875 disk.rbd0.out‚0.078125 disk.sda2.in‚0 disk.sda2.out‚0 network.sent‚ 7.164233e+06 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.rbd2.in‚0afÑæ2”- _wandbruntime‚3268  _runtime‚3268Ï%ÁhÆ•À & lr-AdamW/pg1‚0.0007223223505002621 & lr-AdamW/pg2‚0.0007223223505002621 trainer/global_step‚3599 $ _timestamp‚1.788392512355532e+09 _runtime‚3268.111493209 _step‚289Â-í¬Ë–Å trainer/global_step‚3599 $ _timestamp‚1.788392512355532e+09 _runtime‚3268.111493209 _step‚289 & lr-AdamW/pg1‚0.0007223223505002621 & lr-AdamW/pg2‚0.0007223223505002621¡H¬2—- _wandbruntime‚3268  _runtime‚3268>€hµè'˜âO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2956491264512 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2956295085568 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚238467681692042 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚241325186859008 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _runtime‚3268.116001875 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883925123565662e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚61455974 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚244281481944576 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚244281695162880 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252611584 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2939973276678 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚241325186859008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚244281695162880 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚61455974 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚56843582 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 138059455 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚61455976 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚37619903 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚235527917451264 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3103653982720 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚246353033520128 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚246352756004864 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚241325203898368 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 277515264 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚241325203898368 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚522 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚19223157 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚235527900411904 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2956295085568 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚76626079 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚76603481 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2956491264512 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚76603481 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚37619900 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚244281481944576 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚61455976 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 138082055 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3103401371136 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚238467890727942 trainer/global_step‚3599 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚19223679 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚243249354633728 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚56843057 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2939781280138 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚241325203898368 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚522 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚19223157 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚235527900411904 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2956295085568 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚76626079 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚76603481 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2956491264512 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚76603481 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚37619900 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚244281481944576 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚61455976 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 138082055 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3103401371136 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚238467890727942 trainer/global_step‚3599 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚19223679 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚243249354633728 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚56843057 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2939781280138 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _runtime‚3268.116001875 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 % _timestamp‚1.7883925123565662e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚61455974 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚244281481944576 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚244281695162880 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252611584 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2939973276678 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚241325186859008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚244281695162880 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚61455974 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚56843582 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 138059455 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328¢—åÒ2š- _wandbruntime‚3268  _runtime‚3268ºßc™r›ìM H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚76646977 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2957093476352 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2940575163086 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚241390242935296 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚61472496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚3268.886378214 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2957289668608 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚76646977 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 275404800 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚244347319372288 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 138119473 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3104482844672 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚76624353 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚3599 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚241390242935296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚241390225895936 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚238532074959566 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚235591516835840 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2957289668608 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚56858146 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚243314880130048 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚61472496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚61472494 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19228724 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3104232343552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚241390225895936 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚235591499796480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚246419387878400 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚246419112473600 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚244347532603904 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 138096847 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚37629937 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888‰e§ý÷&œñM K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2957289668608 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚56858146 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚243314880130048 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚61472496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚61472494 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19228724 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3104232343552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚241390225895936 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚235591499796480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚246419387878400 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚246419112473600 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚244347532603904 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 138096847 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚37629937 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚76646977 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2957093476352 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2940575163086 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚241390242935296 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚61472496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚3268.886378214 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2957289668608 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚76646977 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 275404800 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚244347319372288 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 138119473 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3104482844672 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚76624353 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚3599 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚241390242935296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚241390225895936 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚238532074959566 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚235591516835840£l$nÈ2- _wandbruntime‚3268  _runtime‚3268u•{5YžÓ 1loss/FBt1/assignment_loss‚0.11223949768819767 _runtime‚3268.887515216 1loss/FBt2/assignment_loss‚0.20347250545453965 trainer/global_step‚3599 % _timestamp‚1.7883925131316118e+09 1loss/FBt4/assignment_loss‚0.10478289217571728 1loss/SRqqt1/detection_loss‚1.7646803825295396 epoch‚5 /loss/FRt4/assignment_loss‚4.027115616533491 .loss/FBt1/detection_loss‚2.687951202193896 /loss/FBt2/detection_loss‚0.4862972398087181 %loss/total_loss‚45.40245221605463 2loss/SRqqt2/assignment_loss‚1.2455940627447375 1loss/SRqqt3/assignment_loss‚1.673551608528942 0loss/FBt3/assignment_loss‚0.4875540318233626 /loss/FRt1/detection_loss‚2.6254984708719475 /loss/FRt3/detection_loss‚2.6081887464734574 /loss/FRt3/assignment_loss‚3.551663772214817 2loss/SRqqt4/assignment_loss‚1.9096270428236686 .loss/FRt4/detection_loss‚2.609378293156624 1loss/SRqqt2/detection_loss‚0.9798258763926846 1loss/SRqqt4/detection_loss‚2.4529192095453087 .loss/FBt4/detection_loss‚2.575629631678263 1loss/SRqqt1/assignment_loss‚1.606949026563338 0loss/SRqqt3/detection_loss‚2.210491717047989 .loss/FBt3/detection_loss‚2.217470271246774 0loss/FRt2/assignment_loss‚1.8677662114302318 /loss/FRt2/detection_loss‚1.9267183216732173 _step‚292 0loss/FRt1/assignment_loss‚3.4670865854551627YTÙ^ŸØ 1loss/SRqqt2/detection_loss‚0.9798258763926846 1loss/SRqqt4/detection_loss‚2.4529192095453087 .loss/FBt4/detection_loss‚2.575629631678263 1loss/SRqqt1/assignment_loss‚1.606949026563338 0loss/SRqqt3/detection_loss‚2.210491717047989 .loss/FBt3/detection_loss‚2.217470271246774 0loss/FRt2/assignment_loss‚1.8677662114302318 /loss/FRt2/detection_loss‚1.9267183216732173 _step‚292 0loss/FRt1/assignment_loss‚3.4670865854551627 1loss/FBt1/assignment_loss‚0.11223949768819767 _runtime‚3268.887515216 1loss/FBt2/assignment_loss‚0.20347250545453965 trainer/global_step‚3599 % _timestamp‚1.7883925131316118e+09 1loss/FBt4/assignment_loss‚0.10478289217571728 1loss/SRqqt1/detection_loss‚1.7646803825295396 epoch‚5 /loss/FRt4/assignment_loss‚4.027115616533491 .loss/FBt1/detection_loss‚2.687951202193896 /loss/FBt2/detection_loss‚0.4862972398087181 %loss/total_loss‚45.40245221605463 2loss/SRqqt2/assignment_loss‚1.2455940627447375 1loss/SRqqt3/assignment_loss‚1.673551608528942 0loss/FBt3/assignment_loss‚0.4875540318233626 /loss/FRt1/detection_loss‚2.6254984708719475 /loss/FRt3/detection_loss‚2.6081887464734574 /loss/FRt3/assignment_loss‚3.551663772214817 2loss/SRqqt4/assignment_loss‚1.9096270428236686 .loss/FRt4/detection_loss‚2.609378293156624¤iM¨’ ‚(:‡ ÂèâÔ¹¹Õu gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.844' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.5688 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¹`¨N¡‚(:à ÂèâÔŒÑÁx) disk./.usagePercent‚71.53114732049816 disk.rbd3.out‚ 2.1142578125 disk.sda3.out‚ 326.7890625& proc.memory.availableMB‚ 1582.65625 disk.rbd0.out‚0.078125 disk.sda2.out‚0" disk.dm-0.out‚40963.6416015625 disk.dm-1.out‚ 326.7890625 disk.rbd2.in‚0 cpu‚25.432207552424188 proc.cpu.threads‚17 network.recv‚ 1.111295e+06 disk.sda.out‚ 326.7890625 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.in‚0! disk.rbd2.out‚343.67724609375 network.sent‚ 7.169653e+06% disk./.usageGB‚1065.8657875061035 disk.sda1.in‚0 disk.sda.in‚ 0.1171875! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.in‚0 disk.dm-0.in‚ 260.3359375 disk.nvme0n1.in‚ 260.3359375% disk.nvme0n1.out‚40963.6416015625' disk.nvme0c0n1.out‚40963.6416015625 disk.dm-1.in‚ 0.1171875 disk.rbd1.out‚ 1.7763671875# memory_percent‚90.3402328491211 disk.drbd1777.out‚0 disk.rbd3.in‚0 proc.memory.rssMB‚ 2893.71875) proc.memory.percent‚17.66185760498047€•â{¢‚(:„ ÑèâԹܫu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚99.07' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.814 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¥^ G£‚(:¼ ÑèâÔš°´x disk.sda2.out‚0 cpu‚25.433025633098715 network.sent‚ 7.256521e+06 disk.rbd3.out‚ 2.1142578125 disk.drbd1777.in‚0) disk./.usagePercent‚71.53115064859611" disk.dm-0.out‚40964.3564453125 disk.dm-1.out‚339.4375% disk.nvme0n1.out‚40964.3564453125! disk.nvme0c0n1.in‚ 260.3359375 network.recv‚ 1.116528e+06 disk.rbd1.in‚0! disk.rbd2.out‚345.77880859375) proc.memory.percent‚17.66185760498047 proc.cpu.threads‚17$ disk./.usageGB‚1065.865837097168 disk.sda3.in‚ 0.1171875 disk.sda3.out‚339.4375 disk.dm-1.in‚ 0.1171875 disk.nvme0n1.in‚ 260.3359375 disk.sda.out‚339.4375 disk.drbd1777.out‚0$ memory_percent‚90.34028053283691 disk.rbd0.out‚0.078125( proc.memory.availableMB‚ 1582.6484375 disk.rbd3.in‚0 disk.dm-0.in‚ 260.3359375 disk.rbd1.out‚ 1.7763671875 disk.rbd2.in‚0 disk.sda1.out‚0' disk.nvme0c0n1.out‚40964.3564453125 disk.sda2.in‚0 proc.memory.rssMB‚ 2893.71875 disk.sda.in‚ 0.1171875 disk.sda1.in‚0 disk.rbd0.in‚0Ó£MŽ’¤‚(:‡ àèâÔ爵u gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚152.089' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.4178 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0T6áWX¥‚(:Í àèâÔ¾ÙÜx! disk.rbd2.out‚345.77880859375( disk./.usagePercent‚71.5325233609947% disk./.usageGB‚1065.8862915039062 disk.sda.out‚ 351.87109375 disk.dm-1.out‚ 351.87109375 network.sent‚ 7.262636e+06 disk.rbd0.in‚0 disk.rbd0.out‚0.078125* proc.memory.percent‚17.656612396240234 cpu‚25.44932466587124 disk.nvme0n1.in‚ 260.3359375 disk.sda1.out‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 2.1142578125 disk.rbd1.in‚0 proc.cpu.threads‚17 disk.sda.in‚ 0.1171875 disk.dm-0.in‚ 260.3359375$ memory_percent‚90.34647941589355 network.recv‚ 1.121149e+06 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 260.3359375# disk.dm-0.out‚41071.60693359375( proc.memory.availableMB‚ 1581.6328125 disk.sda2.in‚0 disk.sda3.out‚ 351.87109375 disk.rbd1.out‚ 1.83935546875! proc.memory.rssMB‚ 2892.859375& disk.nvme0n1.out‚41071.60693359375 disk.sda1.in‚0( disk.nvme0c0n1.out‚41071.60693359375FëÊC‘¦‚(:† ïèâÔ‰ùÅu gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚161.588' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.3176 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)!%³W§‚(:Ì ïèâÔÍÁÍx disk.sda2.out‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 1.83935546875 disk.rbd2.in‚0* proc.memory.percent‚17.656612396240234 disk.dm-1.out‚ 353.19140625! disk.rbd2.out‚345.79052734375 disk.sda1.out‚0 network.sent‚ 7.269119e+06 disk.sda2.in‚0 disk.dm-1.in‚ 0.1171875 disk.nvme0n1.in‚ 260.3359375) proc.memory.availableMB‚ 1581.68359375 cpu‚25.473822059863355 proc.cpu.threads‚17) disk./.usagePercent‚71.53212142916595% disk./.usageGB‚1065.8803024291992 disk.rbd3.in‚0% disk.nvme0n1.out‚41213.2666015625 disk.sda.out‚ 353.19140625 disk.rbd0.out‚0.078125! proc.memory.rssMB‚ 2892.859375' disk.nvme0c0n1.out‚41213.2666015625 disk.rbd1.in‚0 disk.drbd1777.in‚0 network.recv‚ 1.12576e+06! disk.nvme0c0n1.in‚ 260.3359375 disk.drbd1777.out‚0$ memory_percent‚90.34616947174072 disk.rbd3.out‚ 2.1767578125 disk.dm-0.in‚ 260.3359375 disk.sda.in‚ 0.1171875" disk.dm-0.out‚41213.2666015625 disk.sda3.out‚ 353.19140625 disk.sda1.in‚0 disk.rbd0.in‚0hø%둨‚(:† þèâÔ¹ÄÔu gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚157.279' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.4558 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ëšu¨R©‚(:Ç þèâÔÀˆšy disk.sda2.out‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 cpu‚25.49867696417074 disk.rbd3.in‚0 disk.sda3.out‚ 366.82421875% disk.nvme0n1.out‚41234.2353515625 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.in‚0 disk.sda3.in‚ 0.1171875$ memory_percent‚90.34600257873535) disk./.usagePercent‚71.53248854397005 disk.drbd1777.in‚0' disk.nvme0c0n1.out‚41234.2353515625 disk.sda.out‚ 366.82421875 disk.sda1.out‚0( proc.memory.availableMB‚ 1581.7109375* proc.memory.percent‚17.656612396240234 proc.cpu.threads‚17 disk.nvme0n1.in‚ 260.3359375 disk.sda.in‚ 0.1171875 disk.dm-0.in‚ 260.3359375 disk.rbd1.out‚ 1.90234375 disk.rbd2.in‚0 disk.rbd0.out‚0.140625 network.sent‚ 7.27442e+06 disk.rbd3.out‚ 2.1767578125" disk.dm-0.out‚41234.2353515625! disk.rbd2.out‚345.85302734375 network.recv‚ 1.129363e+06 disk.dm-1.out‚ 366.82421875 disk.sda1.in‚0! proc.memory.rssMB‚ 2892.859375% disk./.usageGB‚1065.8857727050781 disk.drbd1777.out‚0v—+ª2ª- _wandbruntime‚3337  _runtime‚3337ÇñÓŒ$«ý¥ 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9032567739486694 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.14524564146995544 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9117064476013184 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21950455009937286 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19177311658859253 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.37225282192230225 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 7"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.43523970246315 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.34429827332496643 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.43758895993232727 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9988823533058167 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 particle/accuracy_2_of_6‚1 epoch‚5 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9984450936317444 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6071428656578064 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3088180720806122 *jet/accuracy_1_of_3‚0.8097608685493469 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.42989420890808105 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.008102113381028175 jet/accuracy_5_of_6‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.19839683175086975 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.40607765316963196 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.1818556785583496 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3428506553173065 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_1_of_2‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3968149423599243 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.04364679753780365 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.049611229449510574 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 particle/accuracy_1_of_5‚1 particle/accuracy_4_of_5‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.15356387197971344 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2541666328907013 particle/accuracy_1_of_4‚1 particle/accuracy_4_of_6‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 particle/accuracy_3_of_3‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.7666666507720947 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3631802499294281 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_3_of_5‚1 jet/accuracy_4_of_7‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04396825656294823 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.21175114810466766 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04916348308324814 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.17655161023139954 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3571428954601288 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.35983312129974365 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1745050847530365 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4508064389228821 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3571428954601288 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5212243795394897 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.41111108660697937 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.052639178931713104 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.37081414461135864 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4113035500049591 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1513526290655136 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.18171535432338715 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.19005832076072693 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 *jet/accuracy_1_of_5‚0.9375429749488831 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09139776229858398 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.37966978549957275 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.18784475326538086 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.25404849648475647 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1697469800710678 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0029922511894255877 particle/accuracy_5_of_6‚1 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.337281197309494 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 8 PÎJ7,xnurity/*FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 % _timestamp‚1.7883925821398494e+09 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.18571855127811432 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2345990687608719 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.45390212535858154 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974361419677734 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9778704047203064 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.36881500482559204 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19070661067962646 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.988528847694397 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06983944028615952 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07500000298023224 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3359375 particle/accuracy_1_of_7‚1 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.14475411176681519 +jet/accuracy_4_of_6‚0.09649123251438141 particle/accuracy_5_of_7‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29694488644599915 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.21794873476028442 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_2_of_7‚0 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14480002224445343 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.009731634519994259 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.19598709046840668 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.375 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 8"Purity/*FRt0SRqqt*FBt/event_purity‚0.351577490568161 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_3_of_7‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.13333332538604736 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 _step‚293 )loss/val_total_loss‚45.39700600060807 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3706640899181366 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.40783676505088806 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.4888888895511627 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 particle/accuracy_6_of_7‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.18135057389736176 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7280701398849487 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 +particle/specificity‚0.9472941160202026 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3223789930343628 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08695652335882187 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.38487187027931213 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.5350877046585083 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.14228661358356476 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1944444626569748 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.36816707253456116 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 particle/accuracy_7_of_7‚0.5 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.30912694334983826 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 jet/accuracy_6_of_7‚0 trainer/global_step‚3623 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.02127496339380741 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22285063564777374 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4239753186702728 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3480451703071594 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5053379535675049 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1527777761220932 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.14112502336502075 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.18300867080688477 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3932647705078125 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5013492703437805 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9886729717254639 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.34615379571914673 particle/accuracy_6_of_6‚1 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0024811450857669115 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9755951762199402 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014934896491467953 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.43556320667266846 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3675360381603241 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.06862744688987732 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9759466052055359 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3798747956752777 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.19191917777061462 particle/accuracy_1_of_6‚1 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.47472697496414185 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.38572895526885986 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.36242803931236267 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.4329710006713867 *jet/accuracy_1_of_1‚0.4325700104236603 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1849910020828247 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4297080934047699 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3966119587421417 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.42034032940864563 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4038461446762085 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10619967430830002 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1517537534236908 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.17980952560901642 particle/accuracy_1_of_3‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9891083240509033 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.43796294927597046 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42283955216407776 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 'particle/f_score‚0.4038158655166626 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1527777761220932 jet/accuracy_5_of_5‚0 particle/accuracy_3_of_4‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3614393174648285 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.005356325767934322 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8505109548568726 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.41995352506637573 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9968816041946411 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.305515855550766 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7280701398849487 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0902777910232544 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6666666269302368 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8155442476272583 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.13479384779930115 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.09368537366390228 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9919437170028687 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1362241506576538 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.012894957326352596 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9887298941612244 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2787838876247406 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4025941789150238 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4632275402545929 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 +jet/accuracy_3_of_6‚0.14912281930446625 jet/accuracy_7_of_7‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.11741335690021515 +jet/accuracy_2_of_2‚0.09862890094518661 *jet/accuracy_2_of_4‚0.3146189749240875 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9925244450569153 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.59375 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3869359791278839 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5361394286155701 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.36666664481163025 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9919204711914062 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01798376254737377 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9989593625068665 *jet/accuracy_1_of_2‚0.6866920590400696 *jet/accuracy_2_of_3‚0.2059343159198761 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.34682539105415344 *jet/accuracy_1_of_6‚0.9298246502876282 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9088889360427856 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.15979042649269104 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.46224671602249146 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.17592592537403107 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.40242916345596313 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986441135406494 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4566969573497772 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9937373995780945 particle/accuracy_5_of_5‚1 7"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.80952388048172 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995580911636353 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 5 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.47023805975914 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4598214328289032 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,jet/accuracy_4_of_5‚0.008933003060519695 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.019753992557525635 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4499756693840027 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.16980722546577454 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.34624817967414856 particle/accuracy_2_of_7‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8786008358001709 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.39059698581695557 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.37326979637145996 ,jet/accuracy_3_of_3‚0.021000269800424576 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21808543801307678 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13587650656700134 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4357638657093048 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.11094817519187927 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 particle/accuracy_4_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4052259922027588 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27205273509025574 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38007256388664246 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.00685834651812911 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_6‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.07407407462596893 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.11094817519187927 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1828210949897766 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 7"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.34836545586586 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1130952388048172 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3727826774120331 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07765670120716095 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.3932705819606781 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.045719631016254425 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.035555556416511536 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 _runtime‚3337.898135603 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.11087147146463394 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3514172434806824 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_4_of_4‚1 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1334705799818039 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6745418906211853 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8492063879966736 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2678375840187073 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.49984079599380493 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.09298942983150482 +jet/accuracy_3_of_4‚0.05236205458641052 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.29938557744026184 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.39035090804100037 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.40833330154418945 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 (particle/accuracy‚0.8501134514808655 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07758620381355286 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.020611602813005447 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5882353186607361 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9909146428108215 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27954548597335815 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6000000238418579 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9921332597732544 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.38824883103370667 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9925161004066467 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.27291354537010193 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.13320720195770264 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.31909307837486267 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3354733884334564 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.989794909954071 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20663082599639893 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.4285714328289032 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2894294261932373 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9914987087249756 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.986497163772583 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 *particle/sensitivity‚0.325097918510437 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3682473301887512 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.4333333373069763 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.3351820707321167 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.018100358545780182 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.23951038718223572 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3141598403453827 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.1834535151720047 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6420190334320068 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.41995352506637573 ("Purity/1FRt2SRqqt2FBt/event_purity‚0[3Wz¬‚¦ :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.30912694334983826 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 jet/accuracy_6_of_7‚0 trainer/global_step‚3623 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.02127496339380741 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22285063564777374 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.328125 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4239753186702728 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3480451703071594 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5053379535675049 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1527777761220932 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.14112502336502075 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.18300867080688477 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3932647705078125 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5013492703437805 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9886729717254639 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.34615379571914673 particle/accuracy_6_of_6‚1 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0024811450857669115 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9755951762199402 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014934896491467953 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.43556320667266846 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3675360381603241 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.06862744688987732 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9759466052055359 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3798747956752777 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.19191917777061462 particle/accuracy_1_of_6‚1 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.47472697496414185 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.38572895526885986 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.36242803931236267 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.4329710006713867 *jet/accuracy_1_of_1‚0.4325700104236603 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1849910020828247 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4297080934047699 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3966119587421417 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.42034032940864563 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2§…ÁùSRqqt2FBt/event_purity‚0 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4038461446762085 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10619967430830002 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1517537534236908 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.17980952560901642 particle/accuracy_1_of_3‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9891083240509033 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.43796294927597046 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42283955216407776 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 'particle/f_score‚0.4038158655166626 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1527777761220932 jet/accuracy_5_of_5‚0 particle/accuracy_3_of_4‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3614393174648285 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.005356325767934322 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8505109548568726 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.41995352506637573 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9968816041946411 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.305515855550766 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7280701398849487 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0902777910232544 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6666666269302368 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8155442476272583 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.13479384779930115 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.09368537366390228 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9919437170028687 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1362241506576538 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.012894957326352596 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9887298941612244 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2787838876247406 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4025941789150238 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4632275402545929 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 +jet/accuracy_3_of_6‚0.14912281930446625 jet/accuracy_7_of_7‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.11741335690021515 +jet/accuracy_2_of_2‚0.09862890094518661 *jet/accuracy_2_of_4‚0.3146189749240875 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9925244450569153 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.59375 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3869359791278839 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5361394286155701 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.36666664481163025 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9919204711914062 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01798376254737377 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9989593625068665 *jet/accuracy_1_of_2‚0.6866920590400696 *jet/accuracy_2_of_3‚0.2059343159198761 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.34682539105415344 *jet/accuracy_1_of_6‚0.9298246502876282 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9088889360427856 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.15979042649269104 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.46224671602249146 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.17592592537403107 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.40242916345596313 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986441135406494 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4566969573497772 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9937373995780945 particle/accuracy_5_of_5‚1 7"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.80952388048172 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995580911636353 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 5 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.47023805975914 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4598214328289032 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,jet/accuracy_4_of_5‚0.008933003060519695 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.019753992557525635 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4499756693840027 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.16980722546577454 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.34624817967414856 particle/accuracy_2_of_7‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8786008358001709 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.39059698581695557 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.37326979637145996 ,jet/accuracy_3_of_3‚0.021000269800424576 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21808543801307678 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13587650656700134 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4357638657093048 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.11094817519187927 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 particle/accuracy_4_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4052259922027588 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27205273509025574 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.38007256388664246 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.00685834651812911 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_6‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.07407407462596893 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.11094817519187927 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1828210949897766 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 7"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.34836545586586 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.3333333432674408 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1130952388048172 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3727826774120331 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07765670120716095 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.3932705819606781 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.045719631016254425 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.035555556416511536 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 _runtime‚3337.898135603 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.11087147146463394 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3514172434806824 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_4_of_4‚1 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1334705799818039 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6745418906211853 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8492063879966736 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2678375840187073 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.49984079599380493 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.09298942983150482 +jet/accuracy_3_of_4‚0.05236205458641052 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.29938557744026184 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.39035090804100037 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.40833330154418945 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 (particle/accuracy‚0.8501134514808655 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07758620381355286 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.020611602813005447 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5882353186607361 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9909146428108215 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27954548597335815 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6000000238418579 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9921332597732544 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.38824883103370667 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9925161004066467 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.27291354537010193 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.13320720195770264 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.31909307837486267 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3354733884334564 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.989794909954071 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20663082599639893 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.4285714328289032 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2894294261932373 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9914987087249756 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.986497163772583 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 *particle/sensitivity‚0.325097918510437 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3682473301887512 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.4333333373069763 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.3351820707321167 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.018100358545780182 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.23951038718223572 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3141598403453827 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.1834535151720047 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6420190334320068 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.41995352506637573 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9032567739486694 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.14524564146995544 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9117064476013184 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21950455009937286 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19177311658859253 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.37225282192230225 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.3333333432674408 7"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.43523970246315 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.34429827332496643 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.43758895993232727 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9988823533058167 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 particle/accuracy_2_of_6‚1 epoch‚5 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9984450936317444 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6071428656578064 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3088180720806122 *jet/accuracy_1_of_3‚0.8097608685493469 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.42989420890808105 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.008102113381028175 jet/accuracy_5_of_6‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.19839683175086975 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.40607765316963196 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.1818556785583496 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3428506553173065 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_1_of_2‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3968149423599243 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.04364679753780365 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.049611229449510574 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 particle/accuracy_1_of_5‚1 particle/accuracy_4_of_5‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.15356387197971344 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2541666328907013 particle/accuracy_1_of_4‚1 particle/accuracy_4_of_6‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 particle/accuracy_3_of_3‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.7666666507720947 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3631802499294281 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_3_of_5‚1 jet/accuracy_4_of_7‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04396825656294823 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.21175114810466766 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04916348308324814 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.17655161023139954 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3571428954601288 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.35983312129974365 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1745050847530365 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4508064389228821 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3571428954601288 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5212243795394897 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.41111108660697937 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.052639178931713104 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.37081414461135864 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4113035500049591 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1513526290655136 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.18171535432338715 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.19005832076072693 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 *jet/accuracy_1_of_5‚0.9375429749488831 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09139776229858398 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.37966978549957275 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.18784475326538086 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.25404849648475647 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1697469800710678 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0029922511894255877 particle/accuracy_5_of_6‚1 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.337281197309494 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 % _timestamp‚1.7883925821398494e+09 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.18571855127811432 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2345990687608719 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.45390212535858154 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974361419677734 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9778704047203064 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.36881500482559204 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19070661067962646 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.988528847694397 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06983944028615952 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07500000298023224 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3359375 particle/accuracy_1_of_7‚1 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.14475411176681519 +jet/accuracy_4_of_6‚0.09649123251438141 particle/accuracy_5_of_7‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29694488644599915 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.21794873476028442 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_2_of_7‚0 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14480002224445343 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.009731634519994259 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.19598709046840668 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.375 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 8"Purity/*FRt0SRqqt*FBt/event_purity‚0.351577490568161 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_3_of_7‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.13333332538604736 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 _step‚293 )loss/val_total_loss‚45.39700600060807 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3706640899181366 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.40783676505088806 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.4888888895511627 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 particle/accuracy_6_of_7‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.18135057389736176 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7280701398849487 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 +particle/specificity‚0.9472941160202026 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3223789930343628 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08695652335882187 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.38487187027931213 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.5350877046585083 ("Purity/4FRt0SRqqtDBÚæ–0FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.14228661358356476 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1944444626569748 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.36816707253456116 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 particle/accuracy_7_of_7‚0.5¥¡œJœ­‚(:’ éâÔ¯ðÎu gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚100.957' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚20.191399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0~Ú0W®‚(:Ì éâÔôöôx# disk.dm-0.out‚41477.71142578125 network.recv‚ 1.133105e+06) disk./.usagePercent‚71.53147680219473 disk.dm-1.out‚ 380.26953125 network.sent‚ 7.279928e+06 disk.rbd1.in‚0! disk.rbd2.out‚345.85302734375 proc.cpu.threads‚17( disk.nvme0c0n1.out‚41477.71142578125 disk.sda2.in‚0 disk.sda3.in‚ 0.1171875 disk.sda.out‚ 380.26953125 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd3.in‚0" proc.memory.rssMB‚ 2893.7265625) proc.memory.percent‚17.66190528869629 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 260.3359375 disk.rbd3.out‚ 2.2236328125$ memory_percent‚90.35325050354004( proc.memory.availableMB‚ 1580.5234375% disk./.usageGB‚1065.8706970214844 disk.sda3.out‚ 380.26953125 disk.rbd1.out‚ 1.90234375 disk.nvme0n1.in‚ 260.3359375 cpu‚25.510718216625143 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.rbd0.out‚0.140625 disk.dm-0.in‚ 260.3359375& disk.nvme0n1.out‚41477.71142578125 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0i–·¯‚(:„ œéâÔº¢Àu gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚94.93' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.986 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ËqˆV°‚(:Ë œéâÔù®“y disk.sda.out‚ 392.58984375! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.out‚0# disk.dm-0.out‚41601.58251953125$ memory_percent‚90.34714698791504% disk./.usageGB‚1065.8860092163086 disk.rbd3.out‚ 2.2236328125 disk.dm-1.out‚ 392.58984375 disk.sda1.in‚0 disk.drbd1777.in‚0( proc.memory.availableMB‚ 1581.5234375 proc.cpu.threads‚17 network.recv‚ 1.13968e+06 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.sda.in‚ 0.1171875( disk.nvme0c0n1.out‚41601.58251953125 network.sent‚ 7.382922e+06) disk./.usagePercent‚71.53250441643718 disk.sda2.in‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.out‚ 1.90234375 disk.nvme0n1.in‚ 260.3359375 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.dm-0.in‚ 260.3359375 disk.sda3.out‚ 392.58984375 disk.rbd0.out‚0.171875 cpu‚25.511317797090108 disk.sda3.in‚ 0.1171875" proc.memory.rssMB‚ 2892.7265625 disk.rbd2.in‚0! disk.rbd2.out‚345.85302734375 disk.sda1.out‚0) proc.memory.percent‚17.65580177307129& disk.nvme0n1.out‚41601.58251953125‘^r2±- _wandbruntime‚3362  _runtime‚3362GOQeDzÁ _step‚294 & lr-AdamW/pg1‚0.0006978700650335972 & lr-AdamW/pg2‚0.0006978700650335972 trainer/global_step‚3649 % _timestamp‚1.7883926066161463e+09 _runtime‚3362.372125654\DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚249021339857920 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 140596044 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚58033447 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚240017728559104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2995434656756 trainer/global_step‚3649 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚247991903999488 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚62676978 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚38419572 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3161905841152 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚243013355212400 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3012222328832 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚249021553076224 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚19613354 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚246009134568448 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚249021339857920 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚19613872 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚251153809840640 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚240017711519744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚246009117529088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3161663715328 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 140573444 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚77919066 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚247991879095808 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7883926066171794e+09ÌEÀ5í'¶çO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚77896468 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚77919066 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3012418507776 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚38419575 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _runtime‚3362.373400767 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚62676978 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚521 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3012418507776 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚249021339857920 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 140596044 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚58033447 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚240017728559104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2995434656756 trainer/global_step‚3649 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚247991903999488 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚62676978 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚38419572 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3161905841152 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚243013355212400 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3012222328832 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚249021553076224 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚19613354 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚246009134568448 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚249021339857920 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚19613872 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚251153809840640 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚240017711519744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚246009117529088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3161663715328 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 140573444 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚77919066 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚247991879095808 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7883926066171794e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3012222328832 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚62676976 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚58032926 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2995626653296 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚62676976 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚249021553076224 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚518 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚251153542811136 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚246009134568448 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚246009117529088§©ã õ2·- _wandbruntime‚3363  _runtime‚3363­ö€8 ¸ìM UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3013020719616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚251219918226944 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚3363.157523171 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3013216911872 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3013216911872 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚248057429495808 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚58048141 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚519 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚249087177285632 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚58048663 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2996228539704 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚246074156566016 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚522 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚38429609 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚62693496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚19618535 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3162749455872 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚243077539444024 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚38429606 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2996420536452 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 140633462 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚240081327943680 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚249087390517248 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 % _timestamp‚1.7883926074008527e+09 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚249087390517248 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚24JÊƹº6074173605376 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19619054 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚240081310904320 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚296 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚246074173605376 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚62693498 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚77917340 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚77939964 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0‰t¨÷&¹ñM >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚240081327943680 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚249087390517248 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 % _timestamp‚1.7883926074008527e+09 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚249087390517248 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚246074173605376 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19619054 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚240081310904320 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚296 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚246074173605376 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚62693498 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚77917340 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚77939964 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3013020719616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚251219918226944 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚3363.157523171 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3013216911872 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3013216911872 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚248057429495808 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚58048141 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚519 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚249087177285632 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚58048663 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2996228539704 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚246074156566016 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚522 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚38429609 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚62693496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚19618535 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3162749455872 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚243077539444024 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚38429606 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2996420536452 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 140633462 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368¨+@‹2º- _wandbruntime‚3363  _runtime‚3363®ÏnóZ»Ô 2loss/SRqqt3/assignment_loss‚1.4311027214862406 0loss/FBt3/assignment_loss‚0.2550755710353291 1loss/FBt4/assignment_loss‚0.05475957158621615 .loss/FRt1/detection_loss‚2.444435993830363 1loss/SRqqt4/detection_loss‚2.1784803913189816 /loss/FBt2/detection_loss‚0.6038244342697519 .loss/FBt3/detection_loss‚2.191473104451832 /loss/FRt2/assignment_loss‚1.842842363792917 _step‚297 2loss/SRqqt1/assignment_loss‚1.5721146898700835 /loss/FRt2/detection_loss‚1.7383707479533295 /loss/FRt3/detection_loss‚2.5489976036716513 1loss/SRqqt2/detection_loss‚1.0833111193984053 %loss/total_loss‚45.15281930150485 _runtime‚ 3363.15859471 trainer/global_step‚3649 1loss/FBt2/assignment_loss‚0.09445782788417689 epoch‚6 0loss/FRt4/assignment_loss‚4.5607814115862695 2loss/SRqqt2/assignment_loss‚1.1872225227484177 2loss/SRqqt4/assignment_loss‚1.8522235834254668 /loss/FBt1/detection_loss‚2.9425671458244325 .loss/FBt4/detection_loss‚2.966527072282938 /loss/FRt1/assignment_loss‚3.592537734480131 /loss/FRt4/detection_loss‚2.5803611220852023 1loss/SRqqt3/detection_loss‚1.9486468971114268 1loss/FBt1/assignment_loss‚0.15031930433470891 0loss/SRqqt1/detection_loss‚2.045675575891708 $ _timestamp‚1.788392607402786e+09 0loss/FRt3/assignment_loss‚3.2867107911848685ïU¤Ý_¼Ù $ _timestamp‚1.788392607402786e+09 0loss/FRt3/assignment_loss‚3.2867107911848685 2loss/SRqqt3/assignment_loss‚1.4311027214862406 0loss/FBt3/assignment_loss‚0.2550755710353291 1loss/FBt4/assignment_loss‚0.05475957158621615 .loss/FRt1/detection_loss‚2.444435993830363 1loss/SRqqt4/detection_loss‚2.1784803913189816 /loss/FBt2/detection_loss‚0.6038244342697519 .loss/FBt3/detection_loss‚2.191473104451832 /loss/FRt2/assignment_loss‚1.842842363792917 _step‚297 2loss/SRqqt1/assignment_loss‚1.5721146898700835 /loss/FRt2/detection_loss‚1.7383707479533295 /loss/FRt3/detection_loss‚2.5489976036716513 1loss/SRqqt2/detection_loss‚1.0833111193984053 %loss/total_loss‚45.15281930150485 _runtime‚ 3363.15859471 trainer/global_step‚3649 1loss/FBt2/assignment_loss‚0.09445782788417689 epoch‚6 0loss/FRt4/assignment_loss‚4.5607814115862695 2loss/SRqqt2/assignment_loss‚1.1872225227484177 2loss/SRqqt4/assignment_loss‚1.8522235834254668 /loss/FBt1/detection_loss‚2.9425671458244325 .loss/FBt4/detection_loss‚2.966527072282938 /loss/FRt1/assignment_loss‚3.592537734480131 /loss/FRt4/detection_loss‚2.5803611220852023 1loss/SRqqt3/detection_loss‚1.9486468971114268 1loss/FBt1/assignment_loss‚0.15031930433470891 0loss/SRqqt1/detection_loss‚2.045675575891708©ä](j’½‚(:‡ «éâÔð¶²u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.842' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.7684 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0aÌ›P¾‚(:Å «éâÔÇëôw disk.rbd3.in‚0! disk.dm-0.out‚41745.431640625 disk.sda3.out‚ 406.46484375 disk.sda.in‚ 0.1171875 disk.rbd0.out‚0.171875 disk.drbd1777.in‚0 disk.sda.out‚ 406.46484375" proc.memory.rssMB‚ 2892.7265625 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 406.46484375 disk.rbd1.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚346.29052734375! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.in‚0 cpu‚25.511934409498103 network.recv‚ 1.14431e+06% disk./.usageGB‚1065.8982162475586 disk.rbd1.out‚ 1.90234375 proc.cpu.threads‚17) disk./.usagePercent‚71.53332364054674 disk.nvme0n1.in‚ 260.3359375 disk.rbd0.in‚0& disk.nvme0c0n1.out‚41745.431640625$ memory_percent‚90.34585952758789 disk.sda3.in‚ 0.1171875 disk.sda1.in‚0 disk.sda2.out‚0) proc.memory.percent‚17.65580177307129 network.sent‚ 7.389438e+06 disk.dm-0.in‚ 260.3359375$ disk.nvme0n1.out‚41745.431640625 disk.sda1.out‚0' proc.memory.availableMB‚ 1581.734375 disk.rbd3.out‚ 2.28662109375 disk.drbd1777.out‚0-Ø&’¿‚(:‡ ºéâÔÒè¾u gpu.0.gpu‚54.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.493' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.8986 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Úiå3QÀ‚(:Æ ºéâÔô­¸y disk.sda.out‚ 407.578125 disk.rbd3.in‚0 disk.dm-1.in‚ 0.1171875! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.in‚0 disk.dm-0.in‚ 260.3359375 network.sent‚ 7.476016e+06 disk.rbd2.in‚0 disk.dm-1.out‚ 407.578125) proc.memory.availableMB‚ 1581.73828125% disk./.usageGB‚1065.8707733154297 disk.sda1.in‚0 disk.sda3.out‚ 407.578125) proc.memory.percent‚17.65580177307129 cpu‚25.51257557573889 network.recv‚ 1.149635e+06 disk.sda1.out‚0& disk.nvme0c0n1.out‚41852.236328125 disk.sda3.in‚ 0.1171875$ memory_percent‚90.34583568572998 disk.nvme0n1.in‚ 260.3359375 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0 disk.rbd0.out‚ 0.23486328125 disk.drbd1777.in‚0 disk.drbd1777.out‚0$ disk.nvme0n1.out‚41852.236328125 disk.sda2.out‚0 disk.rbd3.out‚ 2.28662109375! disk.dm-0.out‚41852.236328125 proc.cpu.threads‚17) disk./.usagePercent‚71.53148192234542" proc.memory.rssMB‚ 2892.7265625 disk.rbd1.in‚0 disk.rbd1.out‚ 1.90234375! disk.rbd2.out‚346.29052734375pF”öÁ‚(:’ ÉéâÔu gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.847' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.169400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ã’eU‚(:Ê ÉéâÔè£òx disk.sda1.in‚0 network.sent‚ 7.481375e+06! disk.dm-0.out‚41855.376953125 cpu‚25.51302871436593 disk.rbd1.in‚0 disk.sda.out‚ 421.87109375 disk.rbd0.out‚ 0.23486328125& disk.nvme0c0n1.out‚41855.376953125 disk.rbd3.out‚ 2.349609375 disk.dm-1.out‚ 421.87109375 disk.sda1.out‚0% disk./.usageGB‚1065.8708953857422 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 260.3359375 disk.sda3.out‚ 421.87109375 disk.dm-1.in‚ 0.1171875) proc.memory.percent‚17.65580177307129 network.recv‚ 1.153062e+06) disk./.usagePercent‚71.53149011458652$ disk.nvme0n1.out‚41855.376953125 disk.sda3.in‚ 0.1171875" proc.memory.rssMB‚ 2892.7265625 disk.rbd1.out‚ 1.90234375! disk.rbd2.out‚346.36865234375 disk.sda.in‚ 0.1171875 disk.drbd1777.in‚0 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 260.3359375 disk.sda2.out‚0) proc.memory.availableMB‚ 1581.48046875 disk.rbd2.in‚0 disk.nvme0n1.in‚ 260.3359375$ memory_percent‚90.34740924835205 proc.cpu.threads‚17²þ“u2Ã- _wandbruntime‚3405  _runtime‚3405BºŽÇÄÁ _step‚298 & lr-AdamW/pg1‚0.0006735525589169602 & lr-AdamW/pg2‚0.0006735525589169602 trainer/global_step‚3699 % _timestamp‚1.7883926494773004e+09 _runtime‚3405.233271985ÔJé+ÌÅÆ & lr-AdamW/pg2‚0.0006735525589169602 trainer/global_step‚3699 % _timestamp‚1.7883926494773004e+09 _runtime‚3405.233271985 _step‚298 & lr-AdamW/pg1‚0.0006735525589169602ª.g8ª2Æ- _wandbruntime‚3405  _runtime‚3405—³Ëç'ÇáO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚63502978 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 142446396 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3052365875200 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚526 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚523 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚251268178815488 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚63502978 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚252313452290048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3052169696256 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚252313239071744 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3204033823744 _step‚299 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚38921275 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚252313239071744 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚249261069375488 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚252313452290048 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚243197680748544 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚249261069375488 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚78966018 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚254472187735552 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚246232836724708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3052365875200 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚58793878 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚63502976 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚63502976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚78966018 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚249261086414848 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243b…ó½X ÈæO G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚252313239071744 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚249261069375488 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚252313452290048 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚243197680748544 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚249261069375488 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚78966018 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚254472187735552 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚246232836724708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3052365875200 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚58793878 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚63502976 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚63502976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚78966018 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚249261086414848 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚63502978 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 142446396 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3052365875200 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚526 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚523 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚251268178815488 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚63502978 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚252313452290048 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3052169696256 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚252313239071744 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3204033823744 _step‚299 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚38921275 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553«»×'2É- _wandbruntime‚3406  _runtime‚3406&,‡ƒò&ÊìM XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3052968087040 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19877643 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 142483788 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚249326108412416 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚19877123 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3205119709696 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3053164279296 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 % _timestamp‚1.7883926502445943e+09 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚251333679408128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚520 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3204890180096 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚3699 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚300 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚63519498 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚523 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚78986916 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚246297439028340 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3052968087040 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚38931306 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚254538824021504 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3053164279296 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 142483788 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _runtime‚3406.001175445 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚78964292 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚252379076499456 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚58808429 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚246297229992232 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 142506414 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚249326125451776 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚63519496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 142506414 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚38931309 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚249326125451776œ®®¼÷&ËñM WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚63519496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 142506414 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚38931309 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚249326125451776 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3052968087040 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚19877643 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 142483788 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚249326108412416 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚19877123 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3205119709696 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3053164279296 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 % _timestamp‚1.7883926502445943e+09 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚251333679408128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚520 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3204890180096 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 trainer/global_step‚3699 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚300 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚63519498 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚523 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚78986916 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚246297439028340 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3052968087040 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚38931306 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚254538824021504 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3053164279296 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 142483788 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _runtime‚3406.001175445 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚78964292 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚252379076499456 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚58808429 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚246297229992232 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 142506414 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚249326125451776 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553¬'N¾ø2Ì- _wandbruntime‚3406  _runtime‚3406 Ç´PXÍÒ $loss/total_loss‚44.5626250541698 _step‚301 $ _timestamp‚1.788392650246597e+09 0loss/FBt1/assignment_loss‚0.3114463302909616 1loss/SRqqt3/detection_loss‚1.6864421512399401 .loss/FBt3/detection_loss‚2.573861338198185 /loss/FBt4/detection_loss‚2.2169502294725842 .loss/FRt4/detection_loss‚2.303498495983172 0loss/SRqqt1/detection_loss‚1.671353756287776 1loss/SRqqt4/detection_loss‚2.8499676287174225 epoch‚6 2loss/SRqqt4/assignment_loss‚1.8821280734694523 1loss/FBt2/assignment_loss‚0.13048038519250932 1loss/FBt4/assignment_loss‚0.21428752762929232 .loss/FRt1/detection_loss‚2.574588792828413 0loss/FRt2/assignment_loss‚1.7467282343348232 .loss/FRt3/detection_loss‚2.490564402974682 2loss/SRqqt3/assignment_loss‚1.7933901153859637 trainer/global_step‚3699 0loss/FRt3/assignment_loss‚2.9674151319044606 0loss/FRt4/assignment_loss‚4.3292683106434495 2loss/SRqqt1/assignment_loss‚1.8179720323020165 1loss/SRqqt2/detection_loss‚1.0467136235191272 /loss/FBt1/detection_loss‚2.7516096280171323 /loss/FRt1/assignment_loss‚3.548083048600417 1loss/SRqqt2/assignment_loss‚1.199436500740166 0loss/FBt3/assignment_loss‚0.2681335126981139 _runtime‚3406.002435048 /loss/FRt2/detection_loss‚1.6399853175523085 /loss/FBt2/detection_loss‚0.5483204861874321EV)]Î× 2loss/SRqqt1/assignment_loss‚1.8179720323020165 1loss/SRqqt2/detection_loss‚1.0467136235191272 /loss/FBt1/detection_loss‚2.7516096280171323 /loss/FRt1/assignment_loss‚3.548083048600417 1loss/SRqqt2/assignment_loss‚1.199436500740166 0loss/FBt3/assignment_loss‚0.2681335126981139 _runtime‚3406.002435048 /loss/FRt2/detection_loss‚1.6399853175523085 /loss/FBt2/detection_loss‚0.5483204861874321 $loss/total_loss‚44.5626250541698 _step‚301 $ _timestamp‚1.788392650246597e+09 0loss/FBt1/assignment_loss‚0.3114463302909616 1loss/SRqqt3/detection_loss‚1.6864421512399401 .loss/FBt3/detection_loss‚2.573861338198185 /loss/FBt4/detection_loss‚2.2169502294725842 .loss/FRt4/detection_loss‚2.303498495983172 0loss/SRqqt1/detection_loss‚1.671353756287776 1loss/SRqqt4/detection_loss‚2.8499676287174225 epoch‚6 2loss/SRqqt4/assignment_loss‚1.8821280734694523 1loss/FBt2/assignment_loss‚0.13048038519250932 1loss/FBt4/assignment_loss‚0.21428752762929232 .loss/FRt1/detection_loss‚2.574588792828413 0loss/FRt2/assignment_loss‚1.7467282343348232 .loss/FRt3/detection_loss‚2.490564402974682 2loss/SRqqt3/assignment_loss‚1.7933901153859637 trainer/global_step‚3699 0loss/FRt3/assignment_loss‚2.9674151319044606 0loss/FRt4/assignment_loss‚4.3292683106434495­ÃL¢Ï‚(:’ ØéâÔá‡Ñu gpu.0.gpu‚21.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚154.152' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.830399999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0S‰YЂ(:Î ØéâÔ®í—x cpu‚25.51369305608267% disk./.usageGB‚1065.8708953857422% disk.nvme0n1.out‚41953.7412109375 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375" proc.memory.rssMB‚ 2892.7265625! disk.rbd2.out‚346.36865234375 disk.nvme0n1.in‚ 261.26171875 disk.drbd1777.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0' disk.nvme0c0n1.out‚41953.7412109375 disk.rbd3.in‚0 disk.dm-0.in‚ 261.26171875) proc.memory.percent‚17.65580177307129 disk.rbd2.in‚0( proc.memory.availableMB‚ 1581.4453125 disk.dm-1.out‚ 434.24609375 network.sent‚ 7.487534e+06 network.recv‚ 1.157678e+06 disk.sda1.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 1.90234375$ memory_percent‚90.34762382507324 proc.cpu.threads‚17 disk.sda.in‚ 0.1171875 disk.sda.out‚ 434.24609375 disk.rbd0.out‚ 0.2978515625" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.in‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.in‚0 disk.sda3.out‚ 434.24609375 disk.dm-1.in‚ 0.1171875) disk./.usagePercent‚71.53149011458652" disk.dm-0.out‚41953.7412109375zp™Ñ‚(:’ çéâÔÍŽ†v gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.282' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.056400000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0JÁ­\ŠÒ‚(:Ì çéâÔÚ¿ˆx disk.rbd1.in‚0 disk.sda1.out‚0$ memory_percent‚90.34898281097412 proc.cpu.threads‚17( disk./.usagePercent‚71.5314757781646 disk.rbd1.out‚ 1.90234375) proc.memory.availableMB‚ 1581.22265625 network.recv‚ 1.164022e+06% disk./.usageGB‚1065.8706817626953 disk.drbd1777.in‚0" disk.nvme0c0n1.in‚ 261.26171875& disk.nvme0c0n1.out‚42052.224609375 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875! disk.dm-0.out‚42052.224609375 disk.sda3.in‚ 0.1171875 disk.nvme0n1.in‚ 261.26171875) proc.memory.percent‚17.65580177307129 network.sent‚ 7.575338e+06! disk.rbd2.out‚¢P›CÍ346.36865234375 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd3.in‚0 cpu‚25.514131411203905 disk.sda1.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 446.56640625 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.26171875 disk.dm-1.out‚ 446.56640625$ disk.nvme0n1.out‚42052.224609375 disk.sda.out‚ 446.56640625 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625" proc.memory.rssMB‚ 2892.7265625 disk.dm-1.in‚ 0.1171875`™YS2Ó- _wandbruntime‚3448  _runtime‚3448;i¤FÇÔÁ _step‚302 & lr-AdamW/pg1‚0.0006493901355924943 & lr-AdamW/pg2‚0.0006493901355924943 trainer/global_step‚3749 % _timestamp‚1.7883926924268885e+09 _runtime‚3448.182935938g\éÌÕÆ & lr-AdamW/pg2‚0.0006493901355924943 trainer/global_step‚3749 % _timestamp‚1.7883926924268885e+09 _runtime‚3448.182935938 _step‚302 & lr-AdamW/pg1‚0.0006493901355924943®üòŒ2Ö- _wandbruntime‚3448  _runtime‚3448xmmšè'×âO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚20131469 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3246571614208 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚59554441 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚254544428727808 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚79990372 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚64328978 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚252513021221888 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚257791025245696 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚80012970 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚64328978 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚254544453631488 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3092117063680 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3246327391232 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚64328976 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚39422972 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚246377649977344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 144341948 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚249452736308816 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 % _timestamp‚1.7883926924280782e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 144319348 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚255605351503872 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚257790756119040 trainer/global_step‚3749 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3074877295572 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3092117063680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3092313242624 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚80012970 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3075069292112 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚252513038261248 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚252513038261248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚64328976 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚558 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚246377667016704 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚39422975 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚255605138285568 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚252513021221888 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _runtime‚3448.184284308æ Ií'ØçO P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 _runtime‚3448.184284308 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚20131469 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3246571614208 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚59554441 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚254544428727808 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚79990372 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚64328978 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚252513021221888 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚257791025245696 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚80012970 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚64328978 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚254544453631488 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3092117063680 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3246327391232 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚64328976 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚39422972 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚246377649977344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 144341948 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚249452736308816 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 % _timestamp‚1.7883926924280782e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 144319348 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚255605351503872 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚257790756119040 trainer/global_step‚3749 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3074877295572 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3092117063680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3092313242624 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚80012970 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3075069292112 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚252513038261248 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚252513038261248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚64328976 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚558 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚246377667016704 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚39422975 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚255605138285568 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚252513021221888 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130¯¬72Ù- _wandbruntime‚3448  _runtime‚3448‚A}ñ&ÚëM WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚252578060258816 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚257857390661120 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚64345498 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 144356740 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚254609954224128 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚255670975713280 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚80011244 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3093111646720 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚255671188944896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚59570194 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚255671188944896 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚249517129576548 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚246441266401280 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 _step‚304 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚59569678 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3075863175268 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚80011244 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3247411533312 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3247173615104 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚80033868 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚246441249361920 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚64345496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20136672 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 144356740 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚64345498 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3093111646720 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚254609979127808 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚252578077298176 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚39433006 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚257857127839232 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20137185 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚252578060258816 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚39433009 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚252578077298176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3092915454464 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚80033868 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 _runtime‚3448.951530747 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 144379366 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 $ _timestamp‚1.788392693194893e+09ÈXcôþÛðM WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚249517129576548 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚246441266401280 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 _step‚304 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚59569678 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \JDeËJø!viceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3075863175268 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚80011244 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3247411533312 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3247173615104 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚80033868 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚246441249361920 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚64345496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20136672 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 144356740 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚64345498 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3093111646720 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚254609979127808 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚252578077298176 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚39433006 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚257857127839232 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20137185 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚252578060258816 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚39433009 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚252578077298176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3092915454464 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚80033868 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 _runtime‚3448.951530747 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 144379366 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 $ _timestamp‚1.788392693194893e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚252578060258816 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚257857390661120 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚64345498 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 144356740 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚254609954224128 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚255670975713280 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚80011244 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3093111646720 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚255671188944896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚59570194 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚255671188944896 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130°…5Ôè2Ü- _wandbruntime‚3448  _runtime‚3448JùÃ^ÝØ 2loss/SRqqt2/assignment_loss‚1.2931929189795839 .loss/FRt1/detection_loss‚2.423956194552042 /loss/FBt4/detection_loss‚2.3126025515444137 0loss/FRt4/assignment_loss‚3.9185438454151154 0loss/FBt3/assignment_loss‚0.1751466042386662 1loss/SRqqt2/detection_loss‚1.0202141753712992 .loss/FRt3/detection_loss‚2.601456831768155 1loss/SRqqt4/detection_loss‚2.5516278754581103 /loss/FBt3/detection_loss‚2.5107316249295284 epoch‚6 % _timestamp‚1.7883926931968565e+09 0loss/FRt3/assignment_loss‚3.4453833021223543 1loss/SRqqt3/detection_loss‚1.9319179604450862 _runtime‚3448.952653212 0loss/FRt1/assignment_loss‚3.3946345230584503 1loss/FBt1/assignment_loss‚0.09969176861664329 trainer/global_step‚3749 1loss/FBt4/assignment_loss‚0.31410795142394765 1loss/SRqqt1/detection_loss‚1.6675947106037385 /loss/FBt2/detection_loss‚0.6001430300406053 0loss/FRt2/assignment_loss‚1.9643657561391592 2loss/SRqqt1/assignment_loss‚1.4798799032329495 2loss/SRqqt3/assignment_loss‚1.8619155563121916 2loss/SRqqt4/assignment_loss‚1.9032384937459772 .loss/FBt1/detection_loss‚3.290494998296102 %loss/total_loss‚45.15476936763443 _step‚305 0loss/FBt2/assignment_loss‚0.1102135648719536 /loss/FRt2/detection_loss‚1.7901453422382474 /loss/FRt4/detection_loss‚2.4935698842301086qE«wcÞÝ 0loss/FRt2/assignment_loss‚1.9643657561391592 2loss/SRqqt1/assignment_loss‚1.4798799032329495 2loss/SRqqt3/assignment_loss‚1.8619155563121916 2loss/SRqqt4/assignment_loss‚1.9032384937459772 .loss/FBt1/detection_loss‚3.290494998296102 %loss/total_loss‚45.15476936763443 _step‚305 0loss/FBt2/assignment_loss‚0.1102135648719536 /loss/FRt2/detection_loss‚1.7901453422382474 /loss/FRt4/detection_loss‚2.4935698842301086 2loss/SRqqt2/assignment_loss‚1.2931929189795839 .loss/FRt1/detection_loss‚2.423956194552042 /loss/FBt4/detection_loss‚2.3126025515444137 0loss/FRt4/assignment_loss‚3.9185438454151154 0loss/FBt3/assignment_loss‚0.1751466042386662 1loss/SRqqt2/detection_loss‚1.0202141753712992 .loss/FRt3/detection_loss‚2.601456831768155 1loss/SRqqt4/detection_loss‚2.5516278754581103 /loss/FBt3/detection_loss‚2.5107316249295284 epoch‚6 % _timestamp‚1.7883926931968565e+09 0loss/FRt3/assignment_loss‚3.4453833021223543 1loss/SRqqt3/detection_loss‚1.9319179604450862 _runtime‚3448.952653212 0loss/FRt1/assignment_loss‚3.3946345230584503 1loss/FBt1/assignment_loss‚0.09969176861664329 trainer/global_step‚3749 1loss/FBt4/assignment_loss‚0.31410795142394765 1loss/SRqqt1/detection_loss‚1.6675947106037385 /loss/FBt2/detection_loss‚0.6001430300406053±]@ÿß‚(:’ öéâÔªšßu gpu.0.gpu‚21.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.911' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.182199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0C–/N[à‚(:Ð öéâÔ×ÖÕx disk.rbd1.out‚ 1.90234375% disk.nvme0n1.out‚42052.6396484375 disk.sda1.in‚0 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.26171875 disk.sda3.in‚ 0.1171875' disk.nvme0c0n1.out‚42052.6396484375 disk.drbd1777.out‚0" disk.dm-0.out‚42052.6396484375! disk.rbd2.out‚346.36865234375" proc.memory.rssMB‚ 2892.7265625) proc.memory.percent‚17.65580177307129 disk.sda2.out‚0 disk.dm-1.out‚ 447.77734375 proc.cpu.threads‚17 network.recv‚ 1.168044e+06 disk.sda1.out‚0 disk.rbd0.out‚ 0.2978515625" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.in‚0 disk.dm-1.in‚ 0.1171875$ memory_percent‚90.35136699676514 cpu‚25.514695848810153 network.sent‚ 7.580731e+06% disk./.usageGB‚1065.8711585998535 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0) proc.memory.availableMB‚ 1580.83203125 disk.drbd1777.in‚0 disk.rbd3.in‚0) disk./.usagePercent‚71.53150777910638 disk.nvme0n1.in‚ 261.26171875 disk.sda.out‚ 447.77734375 disk.sda3.out‚ 447.77734375 disk.rbd1.in‚0 disk.rbd2.in‚0_Bá‚(:’ …êâÔ°œ«u gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.267' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.453400000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0„¥sêQâ‚(:Æ …êâÔ„ã¢x network.recv‚ 1.173372e+06 disk.drbd1777.in‚0 disk.dm-1.in‚ 0.1171875 cpu‚25.515200241481313 proc.cpu.threads‚17 network.sent‚ 7.667355e+06 disk.sda3.in‚ 0.1171875& disk.nvme0n1.out‚42151.37646484375# disk.dm-0.out‚42151.37646484375 disk.dm-1.out‚ 461.40625 disk.rbd3.out‚ 2.349609375 disk.drbd1777.out‚0 disk.dm-0.in‚ 261.26171875 disk.nvme0n1.in‚ 261.26171875 disk.sda1.in‚0# memory_percent‚90.3498649597168$ disk./.usageGB‚1065.870376586914 disk.sda3.out‚ 461.40625 disk.rbd1.in‚0! disk.rbd2.out‚346.36865234375 disk.sda.out‚ 461.40625" proc.memory.rssMB‚ 2892.7265625 disk.rbd2.in‚0) disk./.usagePercent‚71.53145529756186 disk.rbd0.in‚0" disk.nvme0c0n1.in‚ 261.26171875( disk.nvme0c0n1.out‚42151.37646484375 disk.sda2.in‚0 disk.sda1.out‚0' proc.memory.availableMB‚ 1581.078125 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0 disk.rbd3.in‚0 disk.rbd1.out‚ 1.90234375 disk.sda.in‚ 0.1171875) proc.memory.percent‚17.65580177307129úÚÅ’ã‚(:‡ ”êâÔï—üu gpu.0.gpu‚40.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.088' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.4176 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ãå¿À\ä‚(:Ñ ”êâÔ”¦Šy disk.rbd1.out‚ 1.90234375 cpu‚25.515822096667765 disk.sda2.out‚0# disk.dm-0.out‚42249.73193359375 disk.dm-1.in‚ 0.1171875$ memory_percent‚90.34991264343262 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0 disk.sda3.in‚ 0.1171875 proc.cpu.threads‚17 network.sent‚ 7.672622e+06 disk.sda.out‚ 475.27734375( disk.nvme0c0n1.out‚42249.73193359375 disk.rbd3.in‚0 disk.rbd1.in‚0 network.recv‚ 1.17712e+06) disk./.usagePercent‚71.53147270607418 disk.rbd2.in‚0 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 475.27734375 disk.dm-1.out‚ 475.27734375( proc.memory.availableMB‚ 1581.0703125) proc.memory.percent‚17.65580177307129! disk.rbd2.out‚346.49365234375" proc.memory.rssMB‚ 2892.7265625 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0% disk./.usageGB‚1065.8706359863281 disk.nvme0n1.in‚ 261.26171875& disk.nvme0n1.out‚42249.73193359375 disk.sda.in‚ 0.1171875 disk.sda1.out‚0 disk.rbd0.in‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.dm-0.in‚ 261.26171875л¥:2å- _wandbruntime‚3490  _runtime‚3490]ÀaÆæÀ & lr-AdamW/pg2‚0.0006254029690188964 trainer/global_step‚3799 $ _timestamp‚1.788392734978002e+09 _runtime‚3490.734037467 _step‚306 & lr-AdamW/pg1‚0.0006254029690188964NÇçNËçÅ & lr-AdamW/pg1‚0.0006254029690188964 & lr-AdamW/pg2‚0.0006254029690188964 trainer/global_step‚3799 $ _timestamp‚1.788392734978002e+09 _runtime‚3490.734037467 _step‚306²ROéD2è- _wandbruntime‚3490  _runtime‚3490x/³è'éâO QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚60316581 % _timestamp‚1.7883927349791353e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚258897250717696 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚520 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚258897037499392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚257820703543808 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚258897250717696 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚249557636245504 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚81059922 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚255764973068288 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3288652666368 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚65154976ÆņôêçO N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚258897250717696 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚520 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚258897037499392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚257820703543808 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚258897250717696 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚249557636245504 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚81059922 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚255764973068288 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3288652666368 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚65154976 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚60316581 % _timestamp‚1.7883927349791353e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3133059014144 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚258963088158720 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚81080820 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚261175707772928 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3289733428224 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚255830012105216 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3115392497928 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3132862821888 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚81080820 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 146252318 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚261175987372032 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚255830012105216 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚249621235630080 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚60331258 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚255830029144576 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3289478732800 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚81058196 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3115584494676 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚65171496 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 146252318 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚39934706 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚249621218590720 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚255830029144576 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _runtime‚3491.517327813i÷&íñM TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 146252318 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚261175987372032 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚255830012105216 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚249621235630080 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚60331258 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚255830029144576 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3289478732800 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚81058196 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3115584494676 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚65171496 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 146252318 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚39934706 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚249621218590720 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚255830029144576 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _runtime‚3491.517327813 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3133059014144 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚258963088158720 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚81080820 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚261175707772928 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3289733428224 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚255830012105216 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3115392497928 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3132862821888 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚81080820 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256´jª»Ã2î- _wandbruntime‚3491  _runtime‚3491ÑÞƒ:ZïÔ 0loss/FRt3/assignment_loss‚3.3401804940645086 0loss/FBt3/assignment_loss‚0.1735090112229395 .loss/FBt3/detection_loss‚2.863923416687892 % _timestamp‚1.7883927357629614e+09 /loss/FRt1/assignment_loss‚3.402668889407273 .loss/FRt4/detection_loss‚2.473646342754364 epoch‚6 _runtime‚ 3491.51878541 2loss/SRqqt3/assignment_loss‚1.5490597058467719 2loss/SRqqt4/assignment_loss‚1.9745997895581135 1loss/FBt1/assignment_loss‚0.40570957380207673 1loss/SRqqt3/detection_loss‚1.7825628554790605 -loss/FBt2/detection_loss‚0.65537221061674 /loss/FRt3/detection_loss‚2.4470539820748707 /loss/FBt4/detection_loss‚2.7924267411231996 &loss/total_loss‚45.092787640621324 0loss/FRt2/assignment_loss‚1.9330804544625824 1loss/FBt2/assignment_loss‚0.17409548957081464 1loss/SRqqt4/detection_loss‚2.4501756360565405 _step‚309 2loss/SRqqt1/assignment_loss‚1.5293841361322187 .loss/FRt1/detection_loss‚2.326083235360764 /loss/FRt2/detection_loss‚1.6991691346635525 1loss/SRqqt1/detection_loss‚1.7781683615662836 .loss/FBt1/detection_loss‚2.863111708845411 trainer/global_step‚3799 /loss/FRt4/assignment_loss‚4.188094304955524 2loss/SRqqt2/assignment_loss‚1.2735019556031777 1loss/FBt4/assignment_loss‚0.03381326561793685 1loss/SRqqt2/detection_loss‚0.9833969451487065 9Hï_ðÙ 2loss/SRqqt1/assignment_loss‚1.5293841361322187 .loss/FRt1/detection_loss‚2.326083235360764 /loss/FRt2/detection_loss‚1.6991691346635525 1loss/SRqqt1/detection_loss‚1.7781683615662836 .loss/FBt1/detection_loss‚2.863111708845411 trainer/global_step‚3799 /loss/FRt4/assignment_loss‚4.188094304955524 2loss/SRqqt2/assignment_loss‚1.2735019556031777 1loss/FBt4/assignment_loss‚0.03381326561793685 1loss/SRqqt2/detection_loss‚0.9833969451487065 0loss/FRt3/assignment_loss‚3.3401804940645086 0loss/FBt3/assignment_loss‚0.1735090112229395 .loss/FBt3/detection_loss‚2.863923416687892 % _timestamp‚1.7883927357629614e+09 /loss/FRt1/assignment_loss‚3.402668889407273 .loss/FRt4/detection_loss‚2.473646342754364 epoch‚6 _runtime‚ 3491.51878541 2loss/SRqqt3/assignment_loss‚1.5490597058467719 2loss/SRqqt4/assignment_loss‚1.9745997895581135 1loss/FBt1/assignment_loss‚0.40570957380207673 1loss/SRqqt3/detection_loss‚1.7825628554790605 -loss/FBt2/detection_loss‚0.65537221061674 /loss/FRt3/detection_loss‚2.4470539820748707 /loss/FBt4/detection_loss‚2.7924267411231996 &loss/total_loss‚45.092787640621324 0loss/FRt2/assignment_loss‚1.9330804544625824 1loss/FBt2/assignment_loss‚0.17409548957081464 1loss/SRqqt4/detection_loss‚2.4501756360565405 _step‚309µš·;’ñ‚(:‡ £êâÔ¸ÐÉu gpu.0.gpu‚47.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.622' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.5244 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0†Pò‚(:Å £êâÔ°Õÿw network.recv‚ 1.189254e+06$ disk./.usageGB‚1065.870735168457 disk.nvme0n1.in‚ 261.26171875 disk.sda.out‚ 490.99609375 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 1.90234375# memory_percent‚91.2259578704834 disk.drbd1777.out‚0 disk.dm-1.in‚ 0.1171875( proc.memory.availableMB‚ 1437.5390625" proc.memory.rssMB‚ 2892.7265625 proc.cpu.threads‚17 disk.dm-0.out‚42250.12890625 disk.dm-1.out‚ 490.99609375% disk.nvme0c0n1.out‚42250.12890625 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.26171875 disk.sda3.out‚ 490.99609375) disk./.usagePercent‚71.53147936227008" disk.nvme0c0n1.in‚ 261.26171875 disk.sda3.in‚ 0.1171875 network.sent‚ 7.682998e+06 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.rbd2.in‚0) proc.memory.percent‚17.65580177307129 cpu‚25.5163277099483# disk.nvme0n1.out‚42250.12890625 disk.sda1.out‚0! disk.rbd2.out‚346.49365234375 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0û=·ó‚(:’ ²êâÔ÷àÅu gpu.0.gpu‚31.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.487' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.497400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0/„[Iô‚(:¾ ²êâÔð÷¤y) disk./.usagePercent‚71.53148166633788 disk.rbd3.in‚0 disk.dm-1.out‚492.25 disk.rbd1.out‚ 1.90234375 disk.sda.in‚ 0.1171875" proc.memory.rssMB‚ 2892.7265625 network.sent‚ 7.769585e+06 disk.rbd2.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.drbd1777.in‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd0.out‚ 0.2978515625" disk.nvme0c0n1.in‚ 261.26171875 disk.dm-1.in‚ 0.1171875! disk.rbd2.out‚348.64990234375 disk.rbd3.out‚ 2.349609375) proc.memory.availableMB‚ 1437.54296875) proc.memory.percent‚17.65580177307129% disk.nvme0n1.out‚42348.5458984375 disk.sda1.out‚0' disk.nvme0c0n1.out‚42348.5458984375 disk.sda2.out‚0 proc.cpu.threads‚17% disk./.usageGB‚1065.8707695007324" disk.dm-0.out‚42348.5458984375 disk.sda3.out‚492.25$ memory_percent‚91.22593402862549 disk.sda1.in‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.dm-0.in‚ 261.26171875 cpu‚25.516932897549744 disk.sda.out‚492.25 network.recv‚ 1.194622e+069öRƒ’õ‚(:‡ ÁêâÔ…¥Úu gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.696' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.9392 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Æ«½Wö‚(:Ì ÁêâÔ±ç‹y network.sent‚ 7.774945e+06 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.sda.out‚ 508.04296875 disk.rbd0.out‚ 0.2978515625# memory_percent‚91.2259578704834) proc.memory.percent‚17.65580177307129) disk./.usagePercent‚71.53148166633788 disk.sda1.out‚0 disk.rbd0.in‚0' disk.nvme0c0n1.out‚42349.1982421875 disk.sda2.out‚0 disk.dm-0.in‚ 261.26171875 disk.rbd3.in‚0" disk.dm-0.out‚42349.1982421875" proc.memory.rssMB‚ 2892.7265625 disk.sda3.out‚ 508.04296875 disk.dm-1.out‚ 508.04296875 disk.nvme0n1.in‚ 261.26171875% disk.nvme0n1.out‚42349.1982421875 disk.rbd3.out‚ 2.349609375 network.recv‚ 1.198396e+06 disk.dm-1.in‚ 0.1171875 disk.rbd1.out‚ 1.90234375 disk.drbd1777.in‚0% disk./.usageGB‚1065.8707695007324 disk.rbd1.in‚0 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.in‚0 cpu‚25.517347856998438 proc.cpu.threads‚17 disk.sda3.in‚ 0.1171875 disk.rbd2.out‚ 348.673828125 disk.drbd1777.out‚0( proc.memory.availableMB‚ 1437.5390625ÌÍH|2÷- _wandbruntime‚3533  _runtime‚3533à`ÚŸÇøÁ trainer/global_step‚3849 % _timestamp‚1.7883927779313917e+09 _runtime‚3533.687379903 _step‚310 & lr-AdamW/pg1‚0.0006016110868275512 & lr-AdamW/pg2‚0.0006016110868275512e}bÌùÆ _step‚310 & lr-AdamW/pg1‚0.0006016110868275512 & lr-AdamW/pg2‚0.0006016110868275512 trainer/global_step‚3849 % _timestamp‚1.7883927779313917e+09 _runtime‚3533.687379903¶N92ú- _wandbruntime‚3533  _runtime‚3533ÓSBqÛûâO JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚261096978359808 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚65980978 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6ü>_Á &1078075 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚255891908369332 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚40426375 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3330643235840 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚3849 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚259016941954048 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚264427621595648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 148065252 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3172207977472 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚252737605474304 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 148087852 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚82084276 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚259016924914688 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚82106874 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚61078598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚40426372 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3172011798528 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚262188936713216 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3330881167360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3154511930928 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚20652223 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507Œ¹ë:í'üçO WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3172011798528 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚262188936713216 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3330881167360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3154511930928 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚20652223 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚261096978359808 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚65980978 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚61078075 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚255891908369332 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚40426375 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3330643235840 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚523 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚3849 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚259016941954048 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚264427621595648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 148065252 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3172207977472 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚252737605474304 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 148087852 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚82084276 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚259016924914688 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚82106874 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚61078598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚40426372 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600·…ËNg2ý- _wandbruntime‚3534  _runtime‚3534I0ð&þêM REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3172810189312 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚65997496 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚262254987372544 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚262254774140928 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚61093732 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚259081980990976 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3172810189312 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚255956301636856 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3173006381568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚82127772 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3331487218176 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚252801187819520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚252801204858880 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 148102644 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚497 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3155113817336 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20657323 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 # _timestamp‚1.78839277869494e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚259081963951616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚40436409 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚82105148 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚255956510672964 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚261162528759808 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 148125270 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚3534.451596227 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚65997498 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚65997496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚61093232 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚262254987372544 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚264493991074304 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚259081980990976 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20656826 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 148125270 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚3534.451596227 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚65997498 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚65997496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚61093232 TFDeviceStatsMonitor.on_train_>j4batch_end/inactive_split_bytes.all.current‚ 258627584 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚262254987372544 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚264493991074304 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚259081980990976 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20656826 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3172810189312 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚65997496 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚262254987372544 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚262254774140928 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚61093732 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚259081980990976 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3172810189312 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚255956301636856 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3173006381568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚82127772 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3331487218176 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚252801187819520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚252801204858880 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 148102644 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚500 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚497 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3155113817336 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20657323 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 # _timestamp‚1.78839277869494e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚259081963951616 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚40436409 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750¸*ÊnY2€- _wandbruntime‚3534  _runtime‚3534=XlÙ^Ø trainer/global_step‚3849 0loss/FRt1/assignment_loss‚3.4163856480147814 2loss/SRqqt4/assignment_loss‚2.1181817787928856 1loss/SRqqt1/detection_loss‚1.7029666885888826 1loss/SRqqt2/detection_loss‚1.0083433590883757 1loss/FBt3/assignment_loss‚0.14107453348115087 .loss/FRt4/detection_loss‚2.574395454781396 0loss/SRqqt4/detection_loss‚2.368028846771821 /loss/FRt1/detection_loss‚2.3377293721660153 /loss/FBt1/detection_loss‚2.2360270470380783 1loss/SRqqt3/detection_loss‚1.7288209306869176 /loss/FBt2/detection_loss‚0.6893952855256846 % _timestamp‚1.7883927786969306e+09 .loss/FRt4/assignment_loss‚4.17511357494763 2loss/SRqqt3/assignment_loss‚1.6978044766600768 1loss/FBt1/assignment_loss‚0.16076701906320645 0loss/FBt4/assignment_loss‚0.1738227675523376 2loss/SRqqt1/assignment_loss‚1.4663277774040278 2loss/SRqqt2/assignment_loss‚1.1593053501955892 /loss/FBt3/detection_loss‚2.6576329946517943 .loss/FRt3/detection_loss‚2.376442676143987 /loss/FBt4/detection_loss‚2.9937273301184177 &loss/total_loss‚43.997209603460334 epoch‚6 _step‚313 0loss/FRt2/assignment_loss‚1.8355824773510296 0loss/FRt3/assignment_loss‚3.0697359158879234 1loss/FBt2/assignment_loss‚0.13065225881577683 /loss/FRt2/detection_loss‚1.7789460397325456 _runtime‚3534.452736556Hjc‚Ý 1loss/FBt2/assignment_loss‚0.13065225881577683 /loss/FRt2/detection_loss‚1.7789460397325456 _runtime‚3534.452736556 trainer/global_step‚3849 0loss/FRt1/assignment_loss‚3.4163856480147814 2loss/SRqqt4/assignment_loss‚2.1181817787928856 1loss/SRqqt1/detection_loss‚1.7029666885888826 1loss/SRqqt2/detection_loss‚1.0083433590883757 1loss/FBt3/assignment_loss‚0.14107453348115087 .loss/FRt4/detection_loss‚2.574395454781396 0loss/SRqqt4/detection_loss‚2.368028846771821 /loss/FRt1/detection_loss‚2.3377293721660153 /loss/FBt1/detection_loss‚2.2360270470380783 1loss/SRqqt3/detection_loss‚1.7288209306869176 /loss/FBt2/detection_loss‚0.6893952855256846 % _timestamp‚1.7883927786969306e+09 .loss/FRt4/assignment_loss‚4.17511357494763 2loss/SRqqt3/assignment_loss‚1.6978044766600768 1loss/FBt1/assignment_loss‚0.16076701906320645 0loss/FBt4/assignment_loss‚0.1738227675523376 2loss/SRqqt1/assignment_loss‚1.4663277774040278 2loss/SRqqt2/assignment_loss‚1.1593053501955892 /loss/FBt3/detection_loss‚2.6576329946517943 .loss/FRt3/detection_loss‚2.376442676143987 /loss/FBt4/detection_loss‚2.9937273301184177 &loss/total_loss‚43.997209603460334 epoch‚6 _step‚313 0loss/FRt2/assignment_loss‚1.8355824773510296 0loss/FRt3/assignment_loss‚3.0697359158879234¹µðD´‘ƒ‚(:† ÐêâÔ†Ô¿u gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚96.356' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.2712 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 »ÎM„‚(: ÐêâÔ¦žÝx disk.sda2.in‚0 disk.dm-0.out‚ 42447.640625) proc.memory.availableMB‚ 1437.43359375 disk.sda.in‚ 0.1171875 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 1.90234375 disk.rbd2.out‚ 348.689453125 network.sent‚ 7.781107e+06! disk.nvme0n1.out‚ 42447.640625 disk.sda.out‚ 523.27734375 disk.rbd3.in‚0 proc.cpu.threads‚17 disk.dm-1.in‚ 0.1171875 network.recv‚ 1.202846e+06) disk./.usagePercent‚71.53148166633788% disk./.usageGB‚1065.8707695007324 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375 cpu‚25.517898985173947 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 523.27734375 disk.rbd0.out‚ 0.2978515625" disk.nvme0c0n1.in‚ 261.26171875) proc.memory.percent‚17.65580177307129 disk.sda1.out‚0 disk.dm-0.in‚ 261.26171875 disk.dm-1.out‚ 523.27734375 disk.rbd2.in‚0$ memory_percent‚91.22660160064697 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.26171875# disk.nvme0c0n1.out‚ 42447.640625" proc.memory.rssMB‚ 2892.7265625O‚…‚(:„ ßêâÔ„šÝu gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚104.45' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.89 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0³¯ŠU†‚(:Ê ßêâÔÄÁÁx disk.rbd3.in‚0 disk.rbd2.out‚ 348.751953125% disk.nvme0n1.out‚42546.1376953125 disk.sda.out‚ 524.77734375" disk.dm-0.out‚42546.1376953125 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.26171875 proc.cpu.threads‚17) disk./.usagePercent‚71.53148166633788% disk./.usageGB‚1065.8707695007324 disk.sda2.out‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 524.77734375 disk.rbd1.out‚ 1.90234375 network.recv‚ 1.209014e+06 disk.dm-0.in‚ 261.26171875 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0$ memory_percent‚91.22657775878906 network.sent‚ 7.868813e+06' disk.nvme0c0n1.out‚42546.1376953125 disk.drbd1777.out‚0) proc.memory.percent‚17.65580177307129 cpu‚25.518511981140115 disk.rbd2.in‚0 disk.rbd0.out‚ 0.2978515625% proc.memory.availableMB‚ 1437.4375" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 524.77734375 disk.sda1.in‚0" proc.memory.rssMB‚ 2892.7265625“'‘‡‚(:† îêâÔñ¥¶u gpu.0.gpu‚60.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.595' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.119 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0»ÍLˆ‚(:Á îêâÔºÊøw disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.sda.out‚537.3125 disk.sda2.out‚0" disk.dm-0.out‚42546.4853515625 proc.cpu.threads‚17 network.recv‚ 1.212813e+06 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚42546.4853515625 disk.dm-0.in‚ 261.26171875 cpu‚25.518989017867558 disk.rbd1.out‚ 1.90234375 disk.rbd2.out‚ 348.751953125% disk.nvme0n1.out‚42546.4853515625% disk./.usageGB‚1065.8707695007324 disk.sda1.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0) proc.memory.percent‚17.65580177307129 disk.dm-1.out‚537.3125 disk.sda3.in‚ 0.1171875$ memory_percent‚91.22662544250488( proc.memory.availableMB‚ 1437.4296875 network.sent‚ 7.874144e+06 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚537.3125 disk.rbd1.in‚0 disk.sda.in‚ 0.1171875" disk.nvme0c0n1.in‚ 261.26171875) disk./.usagePercent‚71.53148166633788 disk.sda1.out‚0" proc.memory.rssMB‚ 2892.72656258^„%2‰- _wandbruntime‚3576  _runtime‚3576&¹$ÇŠÁ & lr-AdamW/pg1‚0.0005780343536008392 & lr-AdamW/pg2‚0.0005780343536008392 trainer/global_step‚3899 % _timestamp‚1.7883928205680327e+09 _runtime‚3576.324007752 _step‚314i9@Ì‹Æ & lr-AdamW/pg2‚0.0005780343536008392 trainer/global_step‚3899 % _timestamp‚1.7883928205680327e+09 _runtime‚3576.324007752 _step‚314 & lr-AdamW/pg1‚0.0005780343536008392º¤Ç/ú2Œ- _wandbruntime‚3576  _runtime‚3576Ëó·ç'áO ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚267745922315776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚265480835927040 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 149960804 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚262268876761088 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚61840562 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚264373278079488 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚262268876761088 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3212155344896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚66806978 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚66806976 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚528 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚20912490 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚66806976 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚267746189345280 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚264373253175808 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3372669139968 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3194041253796 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚66806978 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚83131228 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚265480835927040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚40928075 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚40928072 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883928205691023e+09 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚61841090 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚255917557663744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚83131228 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3211959165952 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3194233250336 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚255917574703104 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507Gè¤xìŽæO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚528 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚20912490 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚66806976 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚267746189345280 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚264373253175808 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3372669139968 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3194041253796 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚66806978 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚83131228 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚265480835927040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚40928075 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚40928072 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883928205691023e+09 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚61841090 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚255917557663744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚83131228 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3211959165952 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3194233250336 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚255917574703104 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚267745922315776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/rûø¿ equested_bytes.small_pool.current‚ 191996540 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚265480835927040 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 149960804 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚262268876761088 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚61840562 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚264373278079488 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚262268876761088 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3212155344896 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚66806978 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚66806976 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3»»Ö¢2- _wandbruntime‚3577  _runtime‚3577lûwíð&êM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚40938109 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3212757556736 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3212953748992 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚83152100 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 149998222 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 149998222 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚316 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚262333932837376 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚40938106 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3194835136744 % _timestamp‚1.7883928213344698e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚259176201221172 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚262333915798016 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚262333915798016 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚66823496 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3373754632704 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚83174724 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚267812558208512 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚265546886586368 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚264438803575808 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚265546673354752 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚66823498 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚265546886586368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚83152100 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚66823496 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚61855660 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚83174724 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 3577.0911064 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3212953748992 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚61856184 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20918075 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚259175992185064 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚255981157048320 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚524 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3373512520192 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3195027133492 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20917554 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 149998222 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 149998222 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚316 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚262333932837376 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚40938106 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3194835136744 % _timestamp‚1.7883928213344698e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚259176201221172 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚262333915798016 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚262333915798016 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚66823496 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3373754632704 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚83174724 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚267812558208512 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚265546886586368 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚264438803575808 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚265546673354752 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚66823498 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚265546886586368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚83152100 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚66823496 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚61855660 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚83174724 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _runtime‚ 3577.0911064 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3212953748992 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚61856184 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚20918075 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚259175992185064 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚255981157048320 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚524 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3373512520192 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3195027133492 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚20917554 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3212757556736 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3212953748992 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚83152100 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816¼ÿ“%D2’- _wandbruntime‚3577  _runtime‚3577ÖÐ*³_“Ù 1loss/FBt3/assignment_loss‚0.16891268331424467 /loss/FBt1/detection_loss‚2.3619903050936184 % _timestamp‚1.7883928213363454e+09 /loss/FBt4/detection_loss‚2.3234370164573193 %loss/total_loss‚44.38716333856377 epoch‚6 0loss/FRt2/assignment_loss‚1.6156682325213143 /loss/FRt3/assignment_loss‚3.438700183516457 /loss/FRt4/assignment_loss‚3.992000085204395 2loss/SRqqt3/assignment_loss‚2.0343995869505345 1loss/FBt1/assignment_loss‚0.30062522870478475 1loss/SRqqt1/detection_loss‚1.7732829218203168 1loss/SRqqt3/detection_loss‚1.8466906925452125 1loss/FBt2/assignment_loss‚0.11992278758746266 .loss/FRt4/detection_loss‚2.610455397349685 /loss/FBt2/detection_loss‚0.6090056917457256 1loss/FBt4/assignment_loss‚0.33724213166715344 /loss/FRt3/detection_loss‚2.5107408793909207 1loss/SRqqt2/detection_loss‚1.0741935472114612 _runtime‚3577.092150487 2loss/SRqqt1/assignment_loss‚1.4695010853685297 /loss/FRt2/detection_loss‚1.9054408074262437 _step‚317 1loss/SRqqt4/detection_loss‚2.3833925983598157 0loss/FRt1/assignment_loss‚3.1898523158924554 2loss/SRqqt4/assignment_loss‚1.9836290076134666 /loss/FRt1/detection_loss‚2.4949542175057116 .loss/FBt3/detection_loss‚2.468499775443758 trainer/global_step‚3899 2loss/SRqqt2/assignment_loss‚1.3746261598731755 Ô‡d”Þ % _timestamp‚1.7883928213363454e+09 /loss/FBt4/detection_loss‚2.3234370164573193 %loss/total_loss‚44.38716333856377 epoch‚6 0loss/FRt2/assignment_loss‚1.6156682325213143 /loss/FRt3/assignment_loss‚3.438700183516457 /loss/FRt4/assignment_loss‚3.992000085204395 2loss/SRqqt3/assignment_loss‚2.0343995869505345 1loss/FBt1/assignment_loss‚0.30062522870478475 1loss/SRqqt1/detection_loss‚1.7732829218203168 1loss/SRqqt3/detection_loss‚1.8466906925452125 1loss/FBt2/assignment_loss‚0.11992278758746266 .loss/FRt4/detection_loss‚2.610455397349685 /loss/FBt2/detection_loss‚0.6090056917457256 1loss/FBt4/assignment_loss‚0.33724213166715344 /loss/FRt3/detection_loss‚2.5107408793909207 1loss/SRqqt2/detection_loss‚1.0741935472114612 _runtime‚3577.092150487 2loss/SRqqt1/assignment_loss‚1.4695010853685297 /loss/FRt2/detection_loss‚1.9054408074262437 _step‚317 1loss/SRqqt4/detection_loss‚2.3833925983598157 0loss/FRt1/assignment_loss‚3.1898523158924554 2loss/SRqqt4/assignment_loss‚1.9836290076134666 /loss/FRt1/detection_loss‚2.4949542175057116 .loss/FBt3/detection_loss‚2.468499775443758 trainer/global_step‚3899 2loss/SRqqt2/assignment_loss‚1.3746261598731755 1loss/FBt3/assignment_loss‚0.16891268331424467 /loss/FBt1/detection_loss‚2.3619903050936184½]ƒ™ô’•‚(:‡ ýêâÔ臲u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.197' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.0394 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ª+ÀZ–‚(:Ï ýêâÔ×ûšx disk.sda3.in‚ 0.1171875 disk.sda.out‚ 550.19921875 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625( disk.nvme0c0n1.out‚42644.95361328125 proc.cpu.threads‚17 disk.sda2.out‚0 disk.rbd2.out‚ 348.751953125) proc.memory.availableMB‚ 1437.39453125 disk.nvme0n1.in‚ 261.26171875 disk.rbd3.in‚0 disk.dm-0.in‚ 261.26171875$ disk./.usageGB‚1065.870849609375 disk.sda1.in‚0# disk.dm-0.out‚42644.95361328125 disk.rbd1.in‚0 disk.rbd3.out‚ 2.349609375( disk./.usagePercent‚71.5314870424961 disk.sda.in‚ 0.1171875" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.in‚0 disk.rbd2.in‚0 network.sent‚ 7.879607e+06& disk.nvme0n1.out‚42644.95361328125 disk.drbd1777.out‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 550.19921875 disk.drbd1777.in‚0 network.recv‚ 1.216165e+06 disk.sda1.out‚0 disk.rbd1.out‚ 1.90234375$ memory_percent‚91.22684001922607 disk.sda3.out‚ 550.19921875" proc.memory.rssMB‚ 2892.7265625) proc.memory.percent‚17.65580177307129 cpu‚25.519449979875226"»M ’—‚(:‡ ŒëâÔÊÒ©u gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.672' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.1344 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0p]F˜‚(:» ŒëâÔÉÁ“y disk.sda2.out‚0 disk.nvme0n1.out‚ 42743.3125 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.out‚ 42743.3125 disk.sda2.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.dm-0.out‚ 42743.3125 disk.dm-1.in‚ 0.1171875" proc.memory.rssMB‚ 2892.7265625 proc.cpu.threads‚17 disk.rbd1.in‚0% disk./.usageGB‚1065.8709106445312 disk.drbd1777.in‚0 disk.dm-0.in‚ 261.26171875 disk.dm-1.out‚ 567.35546875 disk.rbd2.in‚0# memory_percent‚91.2268877029419) proc.memory.availableMB‚ 1437.38671875 disk.sda3.out‚ 567.35546875 disk.rbd1.out‚ 1.90234375 network.sent‚ 7.966961e+06 disk.nvme0n1.in‚ 261.26171875 disk.sda.out‚ 567.35546875 disk.rbd2.out‚ 349.888671875) proc.memory.percent‚17.65580177307129" disk.nvme0c0n1.in‚ 261.26171875 disk.sda3.in‚ 0.1171875 cpu‚25.520100231293405 network.recv‚ 1.221867e+06) disk./.usagePercent‚71.53149113861666 disk.sda.in‚ 0.1171875 disk.rbd3.out‚ 2.349609375kFE‚™‚(:’ ›ëâÔó¶×u gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.024' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.804799999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“ Ó‚Pš‚(:Å ›ëâÔÓäˆx# memory_percent‚91.2268877029419 proc.cpu.threads‚17 disk.rbd3.in‚0$ disk.nvme0n1.out‚42744.326171875 disk.sda.in‚ 0.1171875 disk.sda.out‚ 582.1796875 disk.sda3.in‚ 0.1171875 network.sent‚ 7.974771e+06 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375) proc.memory.availableMB‚ 1437.38671875 disk.sda3.out‚ 582.1796875 cpu‚25.52043781916803! disk.dm-0.out‚42744.326171875) disk./.usagePercent‚71.53148985857898 disk.dm-1.out‚ 582.1796875 disk.rbd1.in‚0 disk.drbd1777.out‚0 network.recv‚ 1.228177e+06 disk.rbd2.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.sda1.in‚0 disk.sda2.out‚0 disk.sda1.out‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.dm-1.in‚ 0.1171875 disk.rbd2.out‚ 349.888671875 disk.drbd1777.in‚0" proc.memory.rssMB‚ 2892.7265625) proc.memory.percent‚17.65580177307129 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625$ disk./.usageGB‚1065.870891571045& disk.nvme0c0n1.out‚42744.326171875 disk.dm-0.in‚ 261.26171875 disk.rbd1.out‚ 1.90234375˃2›- _wandbruntime‚3619  _runtime‚3619°@­ÇœÁ trainer/global_step‚3949 % _timestamp‚1.7883928636916418e+09 _runtime‚3619.447613041 _step‚318 & lr-AdamW/pg1‚0.0005546924542865794 & lr-AdamW/pg2‚0.0005546924542865794y™±ÌÆ _runtime‚3619.447613041 _step‚318 & lr-AdamW/pg1‚0.0005546924542865794 & lr-AdamW/pg2‚0.0005546924542865794 trainer/global_step‚3949 % _timestamp‚1.7883928636916418e+09¾WŸ(Ð2ž- _wandbruntime‚3619  _runtime‚3619vw¨˜ÎŸàO VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚271064546375168 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚84200778 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚265520845646848 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚265520828607488 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚67632978 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚556 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚268772948359168 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚84178180 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚259097543931904 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_¶„g segments.current‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚41429772 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3414993479680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚3949 _step‚319 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚67632978 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚271064275151360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3252102712320 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚262331498501648 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚259097526892544 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3251906533376 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3233762573204 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3251906533376 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚265520828607488 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚268772735140864 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚268772735140864 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 151833756 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7883928636926746e+09 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚84178180 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚67632976 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚268772735140864 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 151833756 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7883928636926746e+09 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚84178180 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚67632976 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 JDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚62617641 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚67649498 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚265585884683776 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚268838785800192 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚41439806 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚21177835 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3415831167488 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3252901116416 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3252704924160 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚268838572568576 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚265585867644416 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 $ _timestamp‚1.788392864466168e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚67649496 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚267715078391808 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚67649496 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 151871174 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚259161143316480 trainer/global_step‚3949 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚84221676 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚262395682733272 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3252704924160 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3415593249280 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3234748452900 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚271130909559296 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚259161126277120 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚265585884683776 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚62618180 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚539 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3252901116416 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚271130646737408 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚67649498 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚84199052 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _runtime‚ 3620.22290567 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚84221676 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3252901116416 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3252704924160 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚268838572568576 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚265585867644416 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 $ _timestamp‚1.788392864466168e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚67649496 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚267715078391808 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚67649496 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 151871174 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚259161143316480 trainer/global_step‚3949 NADeviceStatsMonitor.on_train_batch_eë¨^5=nd/active.small_pool.allocated‚84221676 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚262395682733272 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3252704924160 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3415593249280 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3234748452900 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚271130909559296 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚259161126277120 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚265585884683776 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚62618180 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚539 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3252901116416 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚271130646737408 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚67649498 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚84199052 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _runtime‚ 3620.22290567 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚84221676 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚62617641 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚67649498 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚265585884683776 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360À˜Lì2¤- _wandbruntime‚3620  _runtime‚3620 ù`¯[¥Õ .loss/FBt4/detection_loss‚2.780866258284625 %loss/total_loss‚44.81989094673429 1loss/SRqqt2/detection_loss‚1.1344204654053944 1loss/SRqqt4/detection_loss‚2.4230166816375625 epoch‚6 /loss/FBt1/detection_loss‚3.2975023899759566 /loss/FBt2/detection_loss‚0.7281482438586893 % _timestamp‚1.7883928644684718e+09 0loss/FRt3/assignment_loss‚3.3185001941288217 2loss/SRqqt4/assignment_loss‚1.8178101150502621 1loss/FBt1/assignment_loss‚0.17745392277304614 1loss/FBt3/assignment_loss‚0.15237816543077842 0loss/FRt1/assignment_loss‚3.1620164316892625 /loss/FRt2/detection_loss‚1.9141491938578454 _runtime‚3620.224584192 1loss/SRqqt1/detection_loss‚1.7193837774193976 trainer/global_step‚3949 0loss/FRt2/assignment_loss‚1.9291375461377596 2loss/SRqqt3/assignment_loss‚1.6074907610362226 .loss/FRt3/detection_loss‚2.534172198351692 -loss/FRt4/detection_loss‚2.37963622566816 _step‚321 /loss/FRt4/assignment_loss‚4.087538482369603 2loss/SRqqt2/assignment_loss‚1.1170990685747777 1loss/FBt2/assignment_loss‚0.10482391322620635 1loss/SRqqt3/detection_loss‚2.0425042647191067 2loss/SRqqt1/assignment_loss‚1.6256647027936602 1loss/FBt4/assignment_loss‚0.04402627530121476 ,loss/FRt1/detection_loss‚2.3221592015028 /loss/FBt3/detection_loss‚2.3999924675414435ÉC€™`¦Ú /loss/FBt1/detection_loss‚3.2975023899759566 /loss/FBt2/detection_loss‚0.7281482438586893 % _timestamp‚1.7883928644684718e+09 0loss/FRt3/assignment_loss‚3.3185001941288217 2loss/SRqqt4/assignment_loss‚1.8178101150502621 1loss/FBt1/assignment_loss‚0.17745392277304614 1loss/FBt3/assignment_loss‚0.15237816543077842 0loss/FRt1/assignment_loss‚3.1620164316892625 /loss/FRt2/detection_loss‚1.9141491938578454 _runtime‚3620.224584192 1loss/SRqqt1/detection_loss‚1.7193837774193976 trainer/global_step‚3949 0loss/FRt2/assignment_loss‚1.9291375461377596 2loss/SRqqt3/assignment_loss‚1.6074907610362226 .loss/FRt3/detection_loss‚2.534172198351692 -loss/FRt4/detection_loss‚2.37963622566816 _step‚321 /loss/FRt4/assignment_loss‚4.087538482369603 2loss/SRqqt2/assignment_loss‚1.1170990685747777 1loss/FBt2/assignment_loss‚0.10482391322620635 1loss/SRqqt3/detection_loss‚2.0425042647191067 2loss/SRqqt1/assignment_loss‚1.6256647027936602 1loss/FBt4/assignment_loss‚0.04402627530121476 ,loss/FRt1/detection_loss‚2.3221592015028 /loss/FBt3/detection_loss‚2.3999924675414435 .loss/FBt4/detection_loss‚2.780866258284625 %loss/total_loss‚44.81989094673429 1loss/SRqqt2/detection_loss‚1.1344204654053944 1loss/SRqqt4/detection_loss‚2.4230166816375625 epoch‚6ÁŠÚ6§‚(:’ ªëâÔÄÿÇv gpu.0.gpu‚37.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.847' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.169400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XûW¨‚(:Ì ªëâÔðضx" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.out‚0 disk.dm-1.out‚ 582.7578125 disk.rbd0.out‚ 0.2978515625 disk.sda3.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda.out‚ 582.7578125) proc.memory.availableMB‚ 1437.33203125) proc.memory.percent‚17.65580177307129% disk./.usageGB‚1065.8708801269531 disk.rbd3.out‚ 2.349609375& disk.nvme0n1.out‚42843.02294921875" proc.memory.rssMB‚ 2892.7265625 disk.dm-0.in‚ 261.26171875 disk.sda3.out‚ 582.7578125 disk.nvme0n1.in‚ 261.26171875 disk.sda.in‚ 0.1171875 disk.sda1.out‚0 proc.cpu.threads‚17 cpu‚25.521086935091123 network.sent‚ 7.980504e+06 network.recv‚ 1.2325e+06 disk.rbd1.out‚ 1.90234375 disk.rbd0.in‚0$ memory_percent‚91.22722148895264) disk./.usagePercent‚71.53148909055638 disk.sda2.in‚0 disk.rbd3.in‚0 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚42843.02294921875 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd2.out‚ 350.580078125 disk.sda1.in‚0# disk.dm-0.out‚42843.02294921875 disk.drbd1777.in‚0þö’6©‚(:… ¹ëâÔ£Ì×w gpu.0.gpu‚57.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚93.045' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.609 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0h_f³Yª‚(:Î ¹ëâÔÆÝ›x disk.rbd1.out‚ 1.90234375 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd3.in‚0) proc.memory.percent‚17.65580177307129 disk.sda.out‚ 596.30078125 disk.sda2.in‚0" proc.memory.rssMB‚ 2892.7265625 disk.sda2.out‚0 network.sent‚ 8.070508e+06 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 596.30078125 disk.dm-1.out‚ 596.30078125 disk.nvme0n1.in‚ 261.26171875 disk.rbd3.out‚ 2.349609375$ memory_percent‚91.22731685638428# disk.dm-0.out‚42941.61669921875 disk.dm-1.in‚ 0.1171875) proc.memory.availableMB‚ 1437.31640625 cpu‚25.521672498987 proc.cpu.threads‚17 disk.rbd2.out‚ 350.580078125 network.recv‚ 1.244097e+06% disk./.usageGB‚1065.8710403442383 disk.dm-0.in‚ 261.26171875& disk.nvme0n1.out‚42941.61669921875 disk.rbd0.out‚ 0.2978515625 disk.rbd1.in‚0 disk.rbd2.in‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0 disk.drbd1777.out‚0) disk./.usagePercent‚71.53149984287282 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚42941.61669921875´«‚(:… ÈëâÔí¢½u gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚140.51' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.102 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ׂX¬‚(:Í ÈëâÔà‹Ów disk.rbd3.in‚0 disk.dm-1.out‚ 610.3828125" proc.memory.rssMB‚ 2892.7265625 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd1.out‚ 1.90234375 disk.rbd2.in‚0 disk.rbd2.out‚ 350.603515625 disk.nvme0n1.in‚ 261.26171875 cpu‚25.522125513951565 disk.dm-0.in‚ 261.26171875& disk.nvme0n1.out‚42941.96435546875 disk.sda1.in‚0 network.sent‚ 8.077511e+06 disk.sda3.in‚ 0.1171875$ memory_percent‚91.22726917266846 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 610.3828125 disk.sda.in‚ 0.1171875 disk.sda.out‚ 610.3828125) proc.memory.availableMB‚ 1437.32421875 disk.sda1.out‚0 disk.rbd3.out‚ 2.349609375) disk./.usagePercent‚71.53149856283515 disk.sda2.in‚0 network.recv‚ 1.249693e+06 disk.drbd1777.in‚0" disk.nvme0c0n1.in‚ 261.26171875( disk.nvme0c0n1.out‚42941.96435546875# disk.dm-0.out‚42941.96435546875) proc.memory.percent‚17.65580177307129$ disk./.usageGB‚1065.871021270752 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0ŠØk2­- _wandbruntime‚3662  _runtime‚3662)¥_Å®¿ & lr-AdamW/pg1‚0.0005316048777624537 & lr-AdamW/pg2‚0.0005316048777624537 trainer/global_step‚3999 $ _timestamp‚1.788392906455747e+09 _runtime‚ 3662.21171161 _step‚322å ú%Ê¯Ä & lr-AdamW/pg2‚0.0005316048777624537 trainer/global_step‚3999 $ _timestamp‚1.788392906455747e+09 _runtime‚ 3662.21171161 _step‚322 & lr-AdamW/pg1‚0.0005316048777624537Âuðõ2°- _wandbruntime‚3662  _runtime‚3662DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3292050079744 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3291853900800 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 153706708 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚495 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚41931472 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚3999 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 153684108 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚63364330 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚41931475 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚268772780453888 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚272064634354688 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 % _timestamp‚1.7883929064569626e+09 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚68458978 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚68458976 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚21432858 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚274382876493824 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3456812947968 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚262277496121344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚270925802807808 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3291853900800 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚272064847572992 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚268772797493248 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _step‚323 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚272064847572992 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚262277513160704 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚272064634354688 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚274382615755776 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚268772797493248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3273483892612 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3292050079744 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚270925827711488 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚265551189049856 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚63364828 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚68458976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚85247730 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚268772780453888 _runtime‚3662.213149939 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304€d‰‡Æ"²çO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚21432858 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚274382876493824 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3456812947968 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚262277496121344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚270925802807808 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3291853900800 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚272064847572992 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚268772797493248 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 _step‚323 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚272064847572992 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚262277513160704 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚272064634354688 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚274382615755776 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚268772797493248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3273483892612 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3292050079744 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚270925827711488 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚265551189049856 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚63364828 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚68458976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚85247730 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚268772780453888 _runtime‚3662.213149939 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3292050079744 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3291853900800 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 153706708 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚495 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚41931472 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚3999 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 153684108 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚63364330 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚41931475 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚268772780453888 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚272064634354688 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 % _timestamp‚1.7883929064569626e+09 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚68458978 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚68458976 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900ÇlÇ2³- _wandbruntime‚3662  _runtime‚3662JÈ #ñ&´ëM XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3292652291584 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚262341112545280 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚272130685014016 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚68475498 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚501 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚68475498 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚272130685014016 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚68475496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚268837836530176 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3457648748544 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚85246004 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 153721500 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3457890861056 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚85268628 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚63379445 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚498 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 trainer/global_step‚3999 $ _timestamp‚1.788392907232982e+09 _runtime‚3662.989579385 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3292848483840 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚21438437 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚63379946 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 153744126 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 153721500 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚41941506 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚270991353207808 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚265615582317588 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚265615373281480 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚262341095505920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3274469772308 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚268837819490816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750Ç7ö&µðM P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚272130685014016 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚68475496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚268837836530176 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3457648748544 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚85246004 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 153721500 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3457890861056 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚85268628 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚63379445 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚498 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 trainer/global_step‚3999 $ _timestamp‚1.788392907232982e+09 _runtime‚3662.989579385 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3292848483840 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚21438437 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚63379946 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 153744126 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 153721500 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚41941506 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚270991353207808 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚265615582317588 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚265615373281480 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚262341095505920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3274469772308 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚268837819490816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3292652291584 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚262341112545280 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚272130685014016 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚68475498 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚501 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚68475498 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243ÄÇ2¶- _wandbruntime‚3662  _runtime‚3662K \·Ö /loss/FRt1/detection_loss‚2.2580583314312266 /loss/FBt1/detection_loss‚3.5406277009419034 epoch‚6 2loss/SRqqt3/assignment_loss‚1.6649833331505457 1loss/FBt3/assignment_loss‚0.13111270300704864 /loss/FRt2/detection_loss‚1.8489697030805923 /loss/FRt4/detection_loss‚2.5125325828382414 1loss/SRqqt2/detection_loss‚1.0526202832125435 /loss/FBt2/detection_loss‚0.7438080809635561 _runtime‚3662.991250122 2loss/SRqqt2/assignment_loss‚1.2521084230393171 2loss/SRqqt4/assignment_loss‚1.9221253606640263 1loss/FBt4/assignment_loss‚0.08458560011422378 1loss/SRqqt3/detection_loss‚1.6744319715670177 2loss/SRqqt1/assignment_loss‚1.5048681519040839 $ _timestamp‚1.788392907235417e+09 /loss/FRt1/assignment_loss‚3.380645736100826 0loss/FRt2/assignment_loss‚1.6797831762920727 1loss/FBt2/assignment_loss‚0.14785332021760442 /loss/FRt3/detection_loss‚2.3989153124746823 1loss/SRqqt4/detection_loss‚2.0246116677184163 .loss/FBt3/detection_loss‚2.212060465760853 trainer/global_step‚3999 _step‚325 0loss/FRt3/assignment_loss‚2.6284099490869615 /loss/FRt4/assignment_loss‚3.825866941719839 1loss/FBt1/assignment_loss‚0.17941781305333798 1loss/SRqqt1/detection_loss‚1.8475881659984588 +loss/FBt4/detection_loss‚2.533437281847 &loss/total_loss‚43.049422056184376·WÌØa¸Û 0loss/FRt3/assignment_loss‚2.6284099490869615 /loss/FRt4/assignment_loss‚3.825866941719839 1loss/FBt1/assignment_loss‚0.17941781305333798 1loss/SRqqt1/detection_loss‚1.8475881659984588 +loss/FBt4/detection_loss‚2.533437281847 &loss/total_loss‚43.049422056184376 /loss/FRt1/detection_loss‚2.2580583314312266 /loss/FBt1/detection_loss‚3.5406277009419034 epoch‚6 2loss/SRqqt3/assignment_loss‚1.6649833331505457 1loss/FBt3/assignment_loss‚0.13111270300704864 /loss/FRt2/detection_loss‚1.8489697030805923 /loss/FRt4/detection_loss‚2.5125325828382414 1loss/SRqqt2/detection_loss‚1.0526202832125435 /loss/FBt2/detection_loss‚0.7438080809635561 _runtime‚3662.991250122 2loss/SRqqt2/assignment_loss‚1.2521084230393171 2loss/SRqqt4/assignment_loss‚1.9221253606640263 1loss/FBt4/assignment_loss‚0.08458560011422378 1loss/SRqqt3/detection_loss‚1.6744319715670177 2loss/SRqqt1/assignment_loss‚1.5048681519040839 $ _timestamp‚1.788392907235417e+09 /loss/FRt1/assignment_loss‚3.380645736100826 0loss/FRt2/assignment_loss‚1.6797831762920727 1loss/FBt2/assignment_loss‚0.14785332021760442 /loss/FRt3/detection_loss‚2.3989153124746823 1loss/SRqqt4/detection_loss‚2.0246116677184163 .loss/FBt3/detection_loss‚2.212060465760853 trainer/global_step‚3999 _step‚325Åñâç)‘¹‚(:† ×ëâÔ€àÀu gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚93.674' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7348 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0M/»€Xº‚(:Í ×ëâÔÆ©Äx network.sent‚ 8.09801e+06" disk.nvme0c0n1.in‚ 261.26171875 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚43041.12939453125 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd2.in‚0$ memory_percent‚91.22371673583984% disk./.usageGB‚1065.8710441589355 disk.nvme0n1.in‚ 261.26171875 disk.drbd1777.out‚0 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 628.68359375& proc.memory.availableMB‚ 1437.90625" proc.memory.rssMB‚ 2892.7265625 cpu‚25.522620510201637) disk./.usagePercent‚71.53150009888036 disk.dm-1.out‚ 628.68359375 disk.rbd2.out‚ 350.900390625 network.recv‚ 1.282173e+06& disk.nvme0n1.out‚43041.12939453125 disk.sda.out‚ 628.68359375 disk.sda1.out‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 261.26171875 disk.rbd1.out‚ 1.90234375 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.sda2.out‚0 disk.dm-1.in‚ 0.1171875 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625) proc.memory.percent‚17.65580177307129 disk.sda.in‚ 0.1171875 disk.rbd3.out‚ 2.349609375# disk.dm-0.out‚43041.12939453125 íòF’»‚(:‡ æëâÔÚµw gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚136.337' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.2674 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0jˆÓ:W¼‚(:Ì æëâÔð¨Õx) disk./.usagePercent‚71.53149881884268 disk.sda2.out‚0 disk.sda3.out‚ 629.1796875 disk.rbd2.in‚0 disk.sda1.in‚0) proc.memory.percent‚17.65580177307129 cpu‚25.523122206588482 disk.nvme0n1.in‚ 261.26171875 disk.sda1.out‚0& disk.nvme0n1.out‚43139.53857421875# disk.dm-0.out‚43139.53857421875 disk.dm-1.in‚ 0.1171875 disk.rbd2.out‚ 350.900390625 proc.cpu.threads‚17% disk./.usageGB‚1065.8710250854492 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.rbd1.out‚ 1.90234375( disk.nvme0c0n1.out‚43139.53857421875 disk.rbd0.in‚0" proc.memory.rssMB‚ 2892.7265625" disk.nvme0c0n1.in‚ 261.26171875 disk.sda.in‚ 0.1171875 network.sent‚ 8.184496e+06 network.recv‚ 1.287507e+06 disk.drbd1777.out‚0 disk.sda3.in‚ 0.1171875 disk.dm-1.out‚ 629.1796875 disk.rbd0.out‚ 0.2978515625 disk.dm-0.in‚ 261.26171875 disk.rbd1.in‚0$ memory_percent‚91.22228622436523 disk.rbd3.out‚ 2.349609375 disk.sda.out‚ 629.1796875' proc.memory.availableMB‚ 1438.140625 t(2½- _wandbruntime‚3704  _runtime‚3704PHehƾÀ % _timestamp‚1.7883929491398368e+09 _runtime‚ 3704.89591146 _step‚326 & lr-AdamW/pg1‚0.0005087909005641372 & lr-AdamW/pg2‚0.0005087909005641372 trainer/global_step‚4049ÇQ Ë¿Å & lr-AdamW/pg1‚0.0005087909005641372 & lr-AdamW/pg2‚0.0005087909005641372 trainer/global_step‚4049 % _timestamp‚1.7883929491398368e+09 _runtime‚ 3704.89591146 _step‚326ÆÓ}å+2À- _wandbruntime‚3704  _runtime‚3704ð˜ºñšÁáO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚275356533568512 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚69284978 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚268770670562164 $ _timestamp‚1.788392949140967e+09 _step‚327 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚86294682 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚42433175 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3313397208560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚272024732300288 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚265457482389504 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 155557060 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚274202102527488 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚265457465350144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚268770879598064 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚275356533568512 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3331997447168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 250252288 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚69284976 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3331801268224 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚69284978 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚86272084 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3331801268224 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚275356746786816 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚86272084 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚64127054 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚274202077623808 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 155579660 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚86294682 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3313205212020 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750¹]Uì'ÂæO YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚275356746786816 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚86272084 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚64127054 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚274202077623808 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 155579660 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚86294682 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3313205212020 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚275356533568512 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚69284978 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚268770670562164 $ _timestamp‚1.788392949140967e+09 _step‚327 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚86294682 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚42433175 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3313397208560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚272024732300288 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚265457482389504 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 155557060 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚274202102527488 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚265457465350144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚268770879598064 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚275356533568512 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3331997447168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 250252288 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚69284976 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3331801268224 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚69284978 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚86272084 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3331801268224 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0ÇÒtKÄÂ(:’ õëâÔíÔÍu gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚130.927' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.185399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0é©.üQÄ‚(:Æ õëâÔýé†x disk.sda2.out‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 261.26171875" disk.dm-0.out‚43140.1982421875 proc.cpu.threads‚17 disk.nvme0n1.in‚ 261.26171875 disk.sda.out‚ 644.328125 disk.drbd1777.in‚0 disk.rbd2.out‚ 350.904296875) proc.memory.availableMB‚ 1437.65234375) proc.memory.percent‚17.65580177307129 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 261.26171875 network.recv‚ 1.290864e+06 disk.rbd1.in‚0 network.sent‚ 8.189866e+06 disk.dm-1.out‚ 644.328125% disk.nvme0n1.out‚43140.1982421875 disk.sda.in‚ 0.1171875 disk.sda1.out‚0" memory_percent‚91.225266456604" proc.memory.rssMB‚ 2892.7265625 cpu‚25.523630313770585) disk./.usagePercent‚71.53149881884268% disk./.usageGB‚1065.8710250854492 disk.rbd1.out‚ 1.90234375 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 644.328125 disk.rbd2.in‚0' disk.nvme0c0n1.out‚43140.1982421875Ÿï.æ2Å- _wandbruntime‚3705  _runtime‚3705E3Fñ&ÆëM L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚272089771337216 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚272089771337216 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3499909680640 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚86292956 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚274267628023808 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚328 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 155617078 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚42443209 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚268835063829688 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚277767264396800 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚265521064734720 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚42443206 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚21698606 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚64141812 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3313999094968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3332795851264 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚86292956 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 155617078 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3332599659008 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚69301498 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚69301496 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚64142350 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚268835272865796 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748ºxn¨…ÇðM JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚86292956 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 155617078 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3332599659008 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚69301498 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚69301496 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚64142350 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚268835272865796 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚272089771337216 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚272089771337216 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3499909680640 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_bïÓêqatch_end/active_bytes.small_pool.current‚ 196192256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚86292956 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚274267628023808 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚328 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 155617078 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚42443209 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚535 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚268835063829688 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚277767264396800 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚265521064734720 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚42443206 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚21698606 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚64141812 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3313999094968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3332795851264 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0Èb˜2È- _wandbruntime‚3705  _runtime‚3705:\=þWÉÑ 1loss/SRqqt2/assignment_loss‚1.101575387107422 trainer/global_step‚4049 1loss/FBt1/assignment_loss‚0.12117055304250764 1loss/SRqqt3/detection_loss‚1.8001358782492796 .loss/FBt1/detection_loss‚2.858652710914612 /loss/FBt4/detection_loss‚2.7887440363566083 /loss/FRt4/assignment_loss‚3.921406305172074 2loss/SRqqt3/assignment_loss‚1.7612757141729003 0loss/FBt3/assignment_loss‚0.3868117435486056 .loss/FRt2/detection_loss‚1.802368214755666 /loss/FRt4/detection_loss‚2.5507178852255916 0loss/SRqqt4/detection_loss‚2.418851980318626 /loss/FBt2/detection_loss‚0.6518927096957499 -loss/FBt3/detection_loss‚2.53868160645167 0loss/FBt4/assignment_loss‚0.0733893277123343 1loss/SRqqt1/detection_loss‚1.9115567041455575 _step‚329 epoch‚6 0loss/FRt3/assignment_loss‚3.5442589956171373 2loss/SRqqt4/assignment_loss‚1.8413118372360866 1loss/FBt2/assignment_loss‚0.17402208201633249 0loss/SRqqt2/detection_loss‚1.103396434947604 0loss/FRt1/assignment_loss‚3.3579201352596284 0loss/FRt2/assignment_loss‚2.0402660521806455 2loss/SRqqt1/assignment_loss‚1.6218178439533935 .loss/FRt1/detection_loss‚2.295285287797451 .loss/FRt3/detection_loss‚2.477202005596722 &loss/total_loss‚45.142711431474204 % _timestamp‚1.7883929499198549e+09 _runtime‚ 3705.67562766eo—\ÊÖ 1loss/SRqqt2/assignment_loss‚1.101575387107422 trainer/global_step‚4049 1loss/FBt1/assignment_loss‚0.12117055304250764 1loss/SRqqt3/detection_loss‚1.8001358782492796 .loss/FBt1/detection_loss‚2.858652710914612 /loss/FBt4/detection_loss‚2.7887440363566083 /loss/FRt4/assignment_loss‚3.921406305172074 2loss/SRqqt3/assignment_loss‚1.7612757141729003 0loss/FBt3/assignment_loss‚0.3868117435486056 .loss/FRt2/detection_loss‚1.802368214755666 /loss/FRt4/detection_loss‚2.5507178852255916 0loss/SRqqt4/detection_loss‚2.418851980318626 /loss/FBt2/detection_loss‚0.6518927096957499 -loss/FBt3/detection_loss‚2.53868160645167 0loss/FBt4/assignment_loss‚0.0733893277123343 1loss/SRqqt1/detection_loss‚1.9115567041455575 _step‚329 epoch‚6 0loss/FRt3/assignment_loss‚3.5442589956171373 2loss/SRqqt4/assignment_loss‚1.8413118372360866 1loss/FBt2/assignment_loss‚0.17402208201633249 0loss/SRqqt2/detection_loss‚1.103396434947604 0loss/FRt1/assignment_loss‚3.3579201352596284 0loss/FRt2/assignment_loss‚2.0402660521806455 2loss/SRqqt1/assignment_loss‚1.6218178439533935 .loss/FRt1/detection_loss‚2.295285287797451 .loss/FRt3/detection_loss‚2.477202005596722 &loss/total_loss‚45.142711431474204 % _timestamp‚1.7883929499198549e+09 _runtime‚ 3705.67562766É“L?’Ë‚(:‡ „ìâÔΤv gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.041' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.6082 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0â~ìÔSÌ‚(:È „ìâÔ—”™x disk.rbd2.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.rbd0.out‚ 0.2978515625 disk.rbd3.out‚ 2.349609375 disk.sda1.in‚0$ memory_percent‚91.22719764709473 network.sent‚ 8.276435e+06( disk./.usagePercent‚71.5315003548879 disk.rbd0.in‚0" disk.dm-0.out‚43238.5849609375% disk.nvme0n1.out‚43238.5849609375 disk.sda.out‚ 660.953125% disk./.usageGB‚1065.8710479736328 disk.drbd1777.out‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.dm-0.in‚ 261.26171875 disk.sda3.out‚ 660.953125 disk.dm-1.out‚ 660.953125 disk.sda1.out‚0 disk.sda2.out‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0! disk.rbd2.out‚351.42822265625 cpu‚25.524058753495524 proc.cpu.threads‚17 network.recv‚ 1.318148e+06 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda3.in‚ 0.1171875) proc.memory.percent‚17.65580177307129' disk.nvme0c0n1.out‚43238.5849609375" proc.memory.rssMB‚ 2892.7265625 disk.sda.in‚ 0.1171875( proc.memory.availableMB‚ 1437.3359375 disk.rbd3.in‚0 disk.rbd1.out‚ 1.90234375È…µw’Í‚(:‡ “ìâÔΪÈu gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚114.437' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8874 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0f†uO΂(:Ä “ìâÔ©œœy network.sent‚ 8.353353e+06 disk.sda.out‚ 661.56640625" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.out‚0 disk.rbd3.out‚ 2.349609375) proc.memory.percent‚17.65580177307129 disk.nvme0n1.in‚ 261.26171875 network.recv‚ 1.323523e+06 disk.dm-1.in‚ 0.1171875 disk.rbd1.out‚ 1.90234375 disk.rbd2.in‚0 disk.dm-0.out‚ 43239.453125$ memory_percent‚91.22583866119385) proc.memory.availableMB‚ 1437.55859375 disk.rbd0.out‚ 0.2978515625 disk.rbd3.in‚0" proc.memory.rssMB‚ 2892.7265625 cpu‚25.524601664721807 disk.rbd1.in‚0! disk.rbd2.out‚351.42822265625 disk.sda1.in‚0 disk.rbd0.in‚0) disk./.usagePercent‚71.53150086690296% disk./.usageGB‚1065.8710556030273 disk.sda3.in‚ 0.1171875 disk.drbd1777.in‚0 disk.dm-0.in‚ 261.26171875 disk.sda3.out‚ 661.56640625 disk.sda.in‚ 0.1171875 disk.drbd1777.out‚0 disk.sda2.in‚0! disk.nvme0n1.out‚ 43239.453125 disk.dm-1.out‚ 661.56640625 disk.sda1.out‚0# disk.nvme0c0n1.out‚ 43239.453125 proc.cpu.threads‚17ù ]Ð2Ï- _wandbruntime‚3747  _runtime‚3747à¹DgÉÐà ' lr-AdamW/pg2‚0.00048626957079071835 trainer/global_step‚4099 % _timestamp‚1.7883929920385683e+09 _runtime‚3747.794597916 _step‚330 ' lr-AdamW/pg1‚0.00048626957079071835F”K/ÎÑÈ trainer/global_step‚4099 % _timestamp‚1.7883929920385683e+09 _runtime‚3747.794597916 _step‚330 ' lr-AdamW/pg1‚0.00048626957079071835 ' lr-AdamW/pg2‚0.00048626957079071835Ê$®62Ò- _wandbruntime‚3747  _runtime‚3747Qî¾,ç'ÓáO WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚271990361110372 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚277478352439808 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚70110978 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚331 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚533 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚64887881 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚70110978 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚277478377343488 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚87341634 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚42934872 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚530 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3541138661888 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚281019516005376 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 _runtime‚3747.796021788 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚87319036 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚275276701186048 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚42934875 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚87341634 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚278648432782336 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3371748635648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3371944814592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚281019255267328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚281019516005376 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 _runtime‚3747.796021788 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚87319036 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚275276701186048 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚42934875 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚87341634 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚278648432782336 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3371748635648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3371944814592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚281019255267328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚331 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚533 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚64887881 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 235834368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚70110978 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚277478377343488ËMˆäÀ2Õ- _wandbruntime‚3748  _runtime‚3748Ãøó]tÖëM MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚278714270210048 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚64903156 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚42944906 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 157467404 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 157490030 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 157467404 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚272054754377896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 157490030 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚4099 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚70127496 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚277543877936128 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚268701051002880 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚275341740222976 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3372547026432 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚275341723183616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3372743218688 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3372743218688 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚275341723183616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚281085892952064 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3353720414376 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚70127498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3353912411124 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚87339908 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚268701033963520 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚70127498 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3541756388352 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚272054963414004 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _runtime‚3748.567566211 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 $ _timestamp‚1.788392992811006e+09 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚87362532 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚87362532 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3541990112256 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚278714483441664 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚70127496 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚87339908 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚42944909 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0`9Æ©ö&×ðM B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚64903156 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚42944906 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 157467404 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 157490030 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 157467404 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚272054754377896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 157490030 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 trainer/global_step‚4099 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚70127496 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚277543877936128 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚268701051002880 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚275341740222976 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3372547026432 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚275341723183616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3372743218688 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3372743218688 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚275341723183616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚281085892952064 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3353720414376 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚70127498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3353912411124 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚87339908 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚268701033963520 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚70127498 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3541756388352 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚272054963414004 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 _runtime‚3748.567566211 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 $ _timestamp‚1.788392992811006e+09 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚87362532 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚87362532 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3541990112256 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚278714483441664 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚70127496 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚87339908 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚42944909 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚278714270210048 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713ÌÏ|¨¾2Ø- _wandbruntime‚3748  _runtime‚3748±Óv-ZÙÔ /loss/FRt1/detection_loss‚2.2749058261232555 2loss/SRqqt2/assignment_loss‚1.1435967238420355 trainer/global_step‚4099 0loss/FRt3/assignment_loss‚3.2998112230138346 2loss/SRqqt3/assignment_loss‚1.7623589916543647 .loss/FRt3/detection_loss‚2.575103908777237 0loss/FRt2/assignment_loss‚1.7380547608648027 1loss/SRqqt1/detection_loss‚1.6778287571100963 1loss/SRqqt2/detection_loss‚1.0125865881730403 0loss/SRqqt3/detection_loss‚2.010053066106943 .loss/FBt1/detection_loss‚2.724054460014616 2loss/SRqqt1/assignment_loss‚1.8832194241739455 2loss/SRqqt4/assignment_loss‚1.7679974502987332 1loss/FBt2/assignment_loss‚0.11024041438338937 0loss/FBt3/assignment_loss‚0.4898825594500522 1loss/FBt4/assignment_loss‚0.14472589332849325 .loss/FBt3/detection_loss‚2.294664225408009 0loss/FRt4/assignment_loss‚3.7443939530187182 /loss/FRt4/detection_loss‚2.4712912146415977 1loss/SRqqt4/detection_loss‚2.5452356040477753 /loss/FBt2/detection_loss‚0.5176112769445514 epoch‚6 0loss/FRt1/assignment_loss‚3.1049834737237894 %loss/total_loss‚43.60333674600828 _runtime‚3748.568697843 _step‚333 $ _timestamp‚1.788392992812878e+09 0loss/FBt1/assignment_loss‚0.4090738810786466 -loss/FRt2/detection_loss‚1.76494179133858 /loss/FBt4/detection_loss‚2.1367212784917733×or7_ÚÙ 0loss/FRt3/assignment_loss‚3.2998112230138346 2loss/SRqqt3/assignment_loss‚1.7623589916543647 .loss/FRt3/detection_loss‚2.575103908777237 0loss/FRt2/assignment_loss‚1.7380547608648027 1loss/SRqqt1/detection_loss‚1.6778287571100963 1loss/SRqqt2/detection_loss‚1.0125865881730403 0loss/SRqqt3/detection_loss‚2.010053066106943 .loss/FBt1/detection_loss‚2.724054460014616 2loss/SRqqt1/assignment_loss‚1.8832194241739455 2loss/SRqqt4/assignment_loss‚1.7679974502987332 1loss/FBt2/assignment_loss‚0.11024041438338937 0loss/FBt3/assignment_loss‚0.4898825594500522 1loss/FBt4/assignment_loss‚0.14472589332849325 .loss/FBt3/detection_loss‚2.294664225408009 0loss/FRt4/assignment_loss‚3.7443939530187182 /loss/FRt4/detection_loss‚2.4712912146415977 1loss/SRqqt4/detection_loss‚2.5452356040477753 /loss/FBt2/detection_loss‚0.5176112769445514 epoch‚6 0loss/FRt1/assignment_loss‚3.1049834737237894 %loss/total_loss‚43.60333674600828 _runtime‚3748.568697843 _step‚333 $ _timestamp‚1.788392992812878e+09 0loss/FBt1/assignment_loss‚0.4090738810786466 -loss/FRt2/detection_loss‚1.76494179133858 /loss/FBt4/detection_loss‚2.1367212784917733 /loss/FRt1/detection_loss‚2.2749058261232555 2loss/SRqqt2/assignment_loss‚1.1435967238420355 trainer/global_step‚4099Í“RÛ‚(:„ ¢ìâÔ–»u gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.15' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.83 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'éÄUÜ‚(:Ê ¢ìâÔÓËßw disk.sda.in‚ 0.1171875 disk.sda.out‚ 679.50390625 disk.sda1.in‚0) proc.memory.availableMB‚ 1437.05078125 network.recv‚ 1.327415e+06 disk.rbd3.out‚ 2.349609375! disk.rbd2.out‚351.42822265625 disk.sda2.in‚0" proc.memory.rssMB‚ 2892.7265625 cpu‚25.525045065700912 disk.rbd2.in‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0 disk.dm-1.out‚ 679.50390625 disk.sda2.out‚0$ memory_percent‚91.22893810272217 network.sent‚ 8.357907e+06) disk./.usagePercent‚71.53150214694062 disk.dm-0.in‚ 261.26171875 disk.dm-0.out‚43337.73828125 disk.sda1.out‚0 disk.rbd0.in‚0 proc.cpu.threads‚17" disk.nvme0c0n1.in‚ 261.26171875% disk.nvme0c0n1.out‚43337.73828125) proc.memory.percent‚17.65580177307129% disk./.usageGB‚1065.8710746765137 disk.sda3.out‚ 679.50390625 disk.rbd1.out‚ 1.90234375# disk.nvme0n1.out‚43337.73828125 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.sda3.in‚ 0.1171875 žN_Ý‚(:… ±ìâÔ€ñÌu gpu.0.gpu‚54.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.67' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.334 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’ÊÚ]Þ‚(:Ò ±ìâÔŠ“Ëy disk.drbd1777.out‚0# disk.dm-0.out‚43436.88330078125 disk.rbd2.in‚0 disk.rbd3.in‚0! disk.rbd2.out‚351.43212890625 disk.sda2.in‚0 disk.sda3.out‚ 697.10546875& disk.nvme0n1.out‚43436.88330078125 disk.rbd3.out‚ 2.349609375" proc.memory.rssMB‚ 2892.7265625 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.sda.in‚ 0.1171875 disk.sda1.out‚0) disk./.usagePercent‚71.53155232441733% disk./.usageGB‚1065.8718223571777 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚43436.88330078125 disk.dm-0.in‚ 261.26171875 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875) proc.memory.availableMB‚ 1437.44921875) proc.memory.percent‚17.65580177307129 cpu‚25.52550430910796 network.sent‚ 8.444747e+06 disk.sda.out‚ 697.10546875 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.out‚0 disk.dm-1.out‚ 697.10546875$ memory_percent‚91.22650623321533 proc.cpu.threads‚17 disk.rbd1.out‚ 1.90234375 network.recv‚ 1.332372e+06 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625œ‰ÐÄ’ß‚(:‡ ÀìâÔ–¿¿u gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.087' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.2174 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0è£õ[à‚(:Ð ÀìâÔæˆýx disk.sda2.in‚0 disk.sda3.out‚ 717.75390625 disk.sda.in‚ 0.1171875" proc.memory.rssMB‚ 2892.7265625 network.sent‚ 8.45097e+06$ disk./.usageGB‚1065.871841430664 disk.rbd3.out‚ 2.349609375 disk.rbd1.out‚ 1.90234375 disk.sda3.in‚ 0.1171875 disk.rbd1.in‚0 disk.sda.out‚ 717.75390625$ memory_percent‚91.22941493988037 proc.cpu.threads‚17# disk.dm-0.out‚43437.20361328125 disk.dm-1.out‚ 717.75390625) disk./.usagePercent‚71.53155360445501 disk.rbd0.in‚0 disk.drbd1777.out‚0 disk.sda1.in‚0 cpu‚25.525929079244072" disk.nvme0c0n1.in‚ 261.26171875 disk.sda2.out‚0) proc.memory.availableMB‚ 1436.97265625) proc.memory.percent‚17.65580177307129 network.recv‚ 1.33699e+06 disk.rbd0.out‚ 0.2978515625( disk.nvme0c0n1.out‚43437.20361328125 disk.rbd3.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚351.43212890625 disk.nvme0n1.in‚ 261.26171875& disk.nvme0n1.out‚43437.20361328125 disk.drbd1777.in‚0 disk.dm-0.in‚ 261.26171875 disk.dm-1.in‚ 0.1171875 disk.sda1.out‚0šòÙl2á- _wandbruntime‚3790  _runtime‚3790`ÞÕÉâà trainer/global_step‚4149 % _timestamp‚1.7883930345400934e+09 _runtime‚3790.296085619 _step‚334 ' lr-AdamW/pg1‚0.00046405969220084933 ' lr-AdamW/pg2‚0.00046405969220084933ÎN}¯ÎãÈ trainer/global_step‚4149 % _timestamp‚1.7883930345400934e+09 _runtime‚3790.296085619 _step‚334 ' lr-AdamW/pg1‚0.00046405969220084933 ' lr-AdamW/pg2‚0.00046405969220084933Îkr³2ä- _wandbruntime‚3790  _runtime‚3790zd+yåâO F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 159302964 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3411892182016 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚70936976 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚284337766482432 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚3790.301019465 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚70936976 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚278528635993088 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3411696003072 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚70936978 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚281940545214464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚278528653032448 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚281940331996160 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚281940331996160 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3583114322944 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚271817420847104 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚4149 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚278528635993088 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚280754652159488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3411696003072 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚88388586 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚65649362 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚88388586 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚540 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0×ålí'æçO PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚88388586 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚65649362 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚88388586 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚540 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 159302964 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3411892182016 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚70936976 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚284337766482432 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 _runtime‚3790.301019465 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚70936976 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚278528635993088 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3411696003072 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚70936978 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚281940545214464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚278528653032448 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚281940331996160 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚281940331996160 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3583114322944 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚271817420847104 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚4149 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚278528635993088 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚280754652159488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3411696003072 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936Ï¢‹ë2ç- _wandbruntime‚3791  _runtime‚3791~ Úò&èìM KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 159362982 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3412494393856 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3583704210944 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚70953498 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚280820152752128 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 % _timestamp‚1.7883930353137782e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚282006382655488 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚282006169423872 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚278593692069376 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3583944226304 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3393441733784 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚88386860 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22218023 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚284404121882112 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚70953496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 159340356 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3393633730532 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚22218547 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3412494393856 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚271881003192320 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3412690586112 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚278593675030016 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚43446606 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚88409484 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚4149 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 159340356 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚88386860 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚65664629 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 159362982 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚275274444926104 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚70953498 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 159362982 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3412494393856 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3583704210944 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚70953498 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚280820152752128 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 % _timestamp‚1.7883930353137782e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚282006382655488 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚282006169423872 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚278593692069376 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3583944226304 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3393441733784 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚88386860 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22218023 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚284404121882112 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚70953496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 159340356 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3393633730532 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚22218547 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3412494393856 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚271881003192320 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3412690586112 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚278593675030016 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚43446606 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚88409484 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚4149 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 159340356 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚88386860 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚65664629 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 L>DeviceStatsMonitor.on_train_batch_end/allocationéx®íš.all.allocated‚ 159362982 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚275274444926104 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0Ð ãÇ•2ê- _wandbruntime‚3791  _runtime‚3791~ªw6WëÑ /loss/FRt1/assignment_loss‚3.292537577690617 /loss/FRt4/assignment_loss‚4.252279512714922 2loss/SRqqt2/assignment_loss‚0.9973290038605531 2loss/SRqqt4/assignment_loss‚1.9691543772816658 1loss/FBt1/assignment_loss‚0.30739311985798046 0loss/FBt3/assignment_loss‚0.6065663426286645 /loss/FRt4/detection_loss‚2.5579103618337395 1loss/SRqqt1/detection_loss‚1.9609870733693242 0loss/SRqqt3/detection_loss‚2.045095710733602 0loss/SRqqt4/detection_loss‚2.143556088209152 %loss/total_loss‚44.30894583072715 /loss/FRt2/detection_loss‚1.6336165804754603 /loss/FBt1/detection_loss‚2.0100496849592995 .loss/FBt3/detection_loss‚2.250936839315626 _step‚337 1loss/FBt4/assignment_loss‚0.28372260857577203 /loss/FRt3/assignment_loss‚3.648827520894333 1loss/FBt2/assignment_loss‚0.19238965150969323 1loss/SRqqt2/detection_loss‚0.9292549217740694 2loss/SRqqt3/assignment_loss‚1.7672168087474136 /loss/FBt2/detection_loss‚0.5536876040124247 /loss/FBt4/detection_loss‚2.4256371706724167 trainer/global_step‚4149 $ _timestamp‚1.788393035315809e+09 _runtime‚3791.071699055 /loss/FRt2/assignment_loss‚1.575273690440438 .loss/FRt3/detection_loss‚2.730034588508203 epoch‚6 2loss/SRqqt1/assignment_loss‚1.7871113712123285 -loss/FRt1/detection_loss‚2.38837762144945ß-Æç\ìÖ /loss/FRt3/assignment_loss‚3.648827520894333 1loss/FBt2/assignment_loss‚0.19238965150969323 1loss/SRqqt2/detection_loss‚0.9292549217740694 2loss/SRqqt3/assignment_loss‚1.7672168087474136 /loss/FBt2/detection_loss‚0.5536876040124247 /loss/FBt4/detection_loss‚2.4256371706724167 trainer/global_step‚4149 $ _timestamp‚1.788393035315809e+09 _runtime‚3791.071699055 /loss/FRt2/assignment_loss‚1.575273690440438 .loss/FRt3/detection_loss‚2.730034588508203 epoch‚6 2loss/SRqqt1/assignment_loss‚1.7871113712123285 -loss/FRt1/detection_loss‚2.38837762144945 /loss/FRt1/assignment_loss‚3.292537577690617 /loss/FRt4/assignment_loss‚4.252279512714922 2loss/SRqqt2/assignment_loss‚0.9973290038605531 2loss/SRqqt4/assignment_loss‚1.9691543772816658 1loss/FBt1/assignment_loss‚0.30739311985798046 0loss/FBt3/assignment_loss‚0.6065663426286645 /loss/FRt4/detection_loss‚2.5579103618337395 1loss/SRqqt1/detection_loss‚1.9609870733693242 0loss/SRqqt3/detection_loss‚2.045095710733602 0loss/SRqqt4/detection_loss‚2.143556088209152 %loss/total_loss‚44.30894583072715 /loss/FRt2/detection_loss‚1.6336165804754603 /loss/FBt1/detection_loss‚2.0100496849592995 .loss/FBt3/detection_loss‚2.250936839315626 _step‚337 1loss/FBt4/assignment_loss‚0.28372260857577203Ñø…¢’í‚(:‡ ÏìâÔÆɽu gpu.0.gpu‚57.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚103.382' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.6764 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ý!u]î‚(:Ò ÏìâÔø¬îw disk.rbd3.in‚0 disk.nvme0n1.in‚ 261.26171875 disk.rbd2.in‚0 network.recv‚ 1.341831e+06% disk./.usageGB‚1065.8727111816406 disk.sda.out‚ 719.12890625 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda3.out‚ 719.12890625 disk.dm-1.in‚ 0.1171875) disk./.usagePercent‚71.53161197417282 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 1.90234375! disk.rbd2.out‚351.43212890625 proc.cpu.threads‚17 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.26171875 cpu‚25.526357287217834 disk.sda1.out‚0( disk.nvme0c0n1.out‚43535.67041015625& disk.nvme0n1.out‚43535.67041015625 disk.dm-1.out‚ 719.12890625" proc.memory.rssMB‚ 2892.7265625 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0# disk.dm-0.out‚43535.67041015625 disk.rbd1.in‚0( proc.memory.availableMB‚ 1436.4140625 disk.drbd1777.in‚0" disk.nvme0c0n1.in‚ 261.26171875$ memory_percent‚91.23282432556152) proc.memory.percent‚17.65580177307129 network.sent‚ 8.457582e+06Yø< œï‚(:‘ ÞìâÔ–Ž³u gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚89.693' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.938599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0VqÁWð‚(:Ì ÞìâÔº´Ÿx cpu‚25.526889800453805 disk.sda2.in‚0 disk.sda3.out‚ 734.8671875 disk.dm-1.out‚ 734.8671875 disk.rbd1.out‚ 1.90234375" proc.memory.rssMB‚ 2892.7265625 disk.sda3.in‚ 0.1171875 proc.cpu.threads‚17) disk./.usagePercent‚71.53161069413514! disk.rbd2.out‚351.49462890625% disk./.usageGB‚1065.8726921081543 disk.rbd3.out‚ 2.349609375 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.sda.out‚ 734.8671875) proc.memory.percent‚17.65580177307129 disk.dm-0.in‚ 261.26171875 disk.rbd0.out‚ 0.2978515625' disk.nvme0c0n1.out‚43634.0400390625 network.sent‚ 8.544124e+06" disk.dm-0.out‚43634.0400390625 disk.rbd1.in‚0 disk.rbd0.in‚0$ memory_percent‚91.23291969299316( proc.memory.availableMB‚ 1436.3984375 network.recv‚ 1.347285e+06 disk.nvme0n1.in‚ 261.26171875 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.rbd3.in‚0% disk.nvme0n1.out‚43634.0400390625" disk.nvme0c0n1.in‚ 261.26171875ÕFñ‚(:’ íìâÔ»áu gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚138.866' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.773200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0÷(ÃÃRò‚(:Ç íìâÔý½ïx cpu‚25.52735114968014 disk.sda.out‚ 747.4453125 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 747.4453125 disk.rbd2.in‚0! disk.rbd2.out‚351.55712890625) proc.memory.percent‚17.65580177307129 network.sent‚ 8.549495e+06 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0" proc.memory.rssMB‚ 2892.7265625) disk./.usagePercent‚71.53161581428583% disk./.usageGB‚1065.8727684020996 disk.drbd1777.out‚0" disk.dm-0.out‚43634.8173828125 disk.sda3.in‚ 0.1171875 disk.sda1.in‚0 disk.sda3.out‚ 747.4453125$ memory_percent‚91.23458862304688 network.recv‚ 1.350772e+06 disk.sda2.in‚0 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.26171875% disk.nvme0n1.out‚43634.8173828125" disk.nvme0c0n1.in‚ 261.26171875' disk.nvme0c0n1.out‚43634.8173828125 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.26171875$ proc.memory.availableMB‚1436.125 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0 disk.rbd1.out‚ 1.98828125 proc.cpu.threads‚17 disk.rbd3.in‚0 Õë2ó- _wandbruntime‚3833  _runtime‚3833º Ž Åô¿ trainer/global_step‚4199 % _timestamp‚1.7883930772804408e+09 _runtime‚3833.036445096 _step‚338 % lr-AdamW/pg1‚0.000442179808512903 % lr-AdamW/pg2‚0.000442179808512903C× ²ÊõÄ _runtime‚3833.036445096 _step‚338 % lr-AdamW/pg1‚0.000442179808512903 % lr-AdamW/pg2‚0.000442179808512903 trainer/global_step‚4199 % _timestamp‚1.7883930772804408e+09ÒL³42ö- _wandbruntime‚3833  _runtime‚3833trÝðè'÷âO L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚89412940 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚281780604878848 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚285232444428288 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚287656105052672 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3625178077184 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚22472164 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚71762978 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚89412940 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚287655829634560 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 161175916 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚71762976 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚89435538 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚43938272 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3624927562752 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚274997390075904 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚281780587839488 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3451643370496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚285232231209984 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚71762976 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚278429951242688 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚66410984 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3432369170244 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚548 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3451839549440 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚66410436 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 161175916 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚71762976 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚89435538 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚43938272 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3624927562752 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚274997390075904 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚281780587839488 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3451643370496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚285232231209984 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚71762976 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚278429951242688 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚66410984 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3432369170244 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚548 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3451839549440 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚66410436 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 J2ù- _wandbruntime‚3833  _runtime‚3833s{&ß8 úìM cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚284096452471808 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚284096427568128 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚278494344510420 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 161213308 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚89456436 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚285298281869312 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚71779496 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚281845643915776 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 161235934 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚71779496 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚285298068637696 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.alRÅ1ºl.current‚ 277501952 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252598272 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚278494135474312 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3452637953536 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3625763822080 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3433355049940 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3452441761280 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚43948309 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22477306 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 161213308 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚275060972421120 trainer/global_step‚4199 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚89433812 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3452637953536 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚275060989460480 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚71779498 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚287722191390208 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚287722468892160 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚538 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚3833.807385977 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚66425612 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚89433812 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 _step‚340 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚281845626876416 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3433163053192 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 161235934 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚89456436 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108Á)®÷&ûñM D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 161213308 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚89456436 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚285298281869312 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚71779496 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚281845643915776 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 161235934 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚71779496 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚285298068637696 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 277501952 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 252598272 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚278494135474312 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3452637953536 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3625763822080 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3433355049940 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3452441761280 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚43948309 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22477306 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 161213308 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚275060972421120 trainer/global_step‚4199 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚89433812 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3452637953536 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚275060989460480 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚71779498 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚287722191390208 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚287722468892160 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚538 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚3833.807385977 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚66425612 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚89433812 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 _step‚340 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚281845626876416 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3433163053192 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 161235934 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚89456436 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚284096452471808 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚284096427568128 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚278494344510420Ôéy•P2ü- _wandbruntime‚3833  _runtime‚3833ÇÊŽYýÓ /loss/FRt1/assignment_loss‚3.392341552655908 2loss/SRqqt3/assignment_loss‚1.7467139177955686 .loss/FRt4/detection_loss‚2.311983059895666 1loss/SRqqt1/detection_loss‚1.6863195658303225 1loss/SRqqt3/detection_loss‚1.6737866020202636 .loss/FBt3/detection_loss‚2.244600430778835 epoch‚6 trainer/global_step‚4199 _runtime‚3833.809034062 /loss/FRt3/assignment_loss‚3.482776600637554 1loss/SRqqt1/assignment_loss‚1.760140125747197 /loss/FRt2/detection_loss‚1.7229324914915587 1loss/SRqqt2/detection_loss‚0.9794016011780308 /loss/FBt4/detection_loss‚2.9105401039123535 /loss/FRt2/assignment_loss‚1.506410616059457 /loss/FRt4/assignment_loss‚4.162432408646533 2loss/SRqqt2/assignment_loss‚1.2831657897650954 0loss/SRqqt4/detection_loss‚2.197394998660728 .loss/FBt1/detection_loss‚2.782732332454008 % _timestamp‚1.7883930780527644e+09 /loss/FRt1/detection_loss‚2.4755238211607633 &loss/total_loss‚43.816506509296104 1loss/SRqqt4/assignment_loss‚1.883434083479554 1loss/FBt1/assignment_loss‚0.07358387066963534 1loss/FBt4/assignment_loss‚0.12064667189487668 /loss/FRt3/detection_loss‚2.5760793616006405 0loss/FBt2/detection_loss‚0.47441936262178325 _step‚341 /loss/FBt2/assignment_loss‚0.158740730837575 1loss/FBt3/assignment_loss‚0.21040640950219391×n@›^þØ /loss/FRt3/detection_loss‚2.5760793616006405 0loss/FBt2/detection_loss‚0.47441936262178325 _step‚341 /loss/FBt2/assignment_loss‚0.158740730837575 1loss/FBt3/assignment_loss‚0.21040640950219391 /loss/FRt1/assignment_loss‚3.392341552655908 2loss/SRqqt3/assignment_loss‚1.7467139177955686 .loss/FRt4/detection_loss‚2.311983059895666 1loss/SRqqt1/detection_loss‚1.6863195658303225 1loss/SRqqt3/detection_loss‚1.6737866020202636 .loss/FBt3/detection_loss‚2.244600430778835 epoch‚6 trainer/global_step‚4199 _runtime‚3833.809034062 /loss/FRt3/assignment_loss‚3.482776600637554 1loss/SRqqt1/assignment_loss‚1.760140125747197 /loss/FRt2/detection_loss‚1.7229324914915587 1loss/SRqqt2/detection_loss‚0.9794016011780308 /loss/FBt4/detection_loss‚2.9105401039123535 /loss/FRt2/assignment_loss‚1.506410616059457 /loss/FRt4/assignment_loss‚4.162432408646533 2loss/SRqqt2/assignment_loss‚1.2831657897650954 0loss/SRqqt4/detection_loss‚2.197394998660728 .loss/FBt1/detection_loss‚2.782732332454008 % _timestamp‚1.7883930780527644e+09 /loss/FRt1/detection_loss‚2.4755238211607633 &loss/total_loss‚43.816506509296104 1loss/SRqqt4/assignment_loss‚1.883434083479554 1loss/FBt1/assignment_loss‚0.07358387066963534 1loss/FBt4/assignment_loss‚0.12064667189487668Õ€4fm’ÿ‚(:‡ üìâÔÙÞ°u gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.372' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.8744 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ó«ÀZ€‚(:Ï üìâÔóàÒx# disk.dm-0.out‚43733.30517578125 network.sent‚ 8.555674e+06) disk./.usagePercent‚71.53161453424815% disk./.usageGB‚1065.8727493286133& disk.nvme0n1.out‚43733.30517578125 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0$ memory_percent‚91.23861789703369 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.rbd3.out‚ 2.349609375 proc.cpu.threads‚17 network.recv‚ 1.355364e+06 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda2.out‚0 cpu‚25.52782068773423 disk.rbd1.out‚ 2.05078125( disk.nvme0c0n1.out‚43733.30517578125 disk.rbd2.out‚ 351.88671875 disk.rbd0.in‚0" disk.nvme0c0n1.in‚ 261.26953125 disk.rbd3.in‚0) proc.memory.availableMB‚ 1435.46484375" proc.memory.rssMB‚ 2892.7265625) proc.memory.percent‚17.65580177307129 disk.dm-1.out‚ 760.93359375 disk.rbd1.in‚0 disk.drbd1777.out‚0 disk.sda3.out‚ 760.93359375 disk.sda.out‚ 760.93359375 disk.sda3.in‚ 0.1171875 disk.nvme0n1.in‚ 261.26953125 disk.dm-0.in‚ 261.26953125LLv<’‚(:‡ ‹íâÔ›Õ¥u gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.653' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.3306 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0§R1T‚‚(:É ‹íâÔËî§x disk.dm-1.out‚ 775.43359375" disk.nvme0c0n1.in‚ 261.26953125& disk.nvme0c0n1.out‚43734.255859375) proc.memory.availableMB‚ 1435.94140625 cpu‚25.528161871369992 proc.cpu.threads‚17 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625$ disk.nvme0n1.out‚43734.255859375 disk.drbd1777.out‚0 disk.sda2.out‚0! disk.dm-0.out‚43734.255859375$ memory_percent‚91.23570919036865 network.sent‚ 8.643412e+06 disk.sda.in‚ 0.1171875 disk.rbd3.in‚0 disk.dm-0.in‚ 261.26953125) proc.memory.percent‚17.65580177307129$ disk./.usageGB‚1065.872989654541 disk.rbd1.out‚ 2.05078125 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375 network.recv‚ 1.361663e+06 disk.sda.out‚ 775.43359375" proc.memory.rssMB‚ 2892.7265625 disk.rbd2.in‚0 disk.nvme0n1.in‚ 261.26953125 disk.sda1.out‚0 disk.drbd1777.in‚0) disk./.usagePercent‚71.53163066272282 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd2.out‚ 351.88671875 disk.sda3.out‚ 775.43359375ËClÏ‘ƒ‚(:† šíâÔŠ‚»u gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚153.974' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.7948 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0cÙ‹·Z„‚(:Ï šíâÔœ»”y disk.sda.in‚ 0.1171875) disk./.usagePercent‚71.53166778381528( disk.nvme0c0n1.out‚43832.71435546875 disk.rbd3.out‚ 2.349609375# disk.dm-0.out‚43832.71435546875 disk.nvme0n1.in‚ 261.28515625% disk./.usageGB‚1065.8735427856445 disk.drbd1777.in‚0 disk.rbd3.in‚0' proc.memory.availableMB‚ 1434.109375* proc.memory.percent‚17.666316032409668 network.sent‚ 8.648719e+06 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.dm-1.out‚ 775.9921875 cpu‚25.547732861304215 disk.sda3.out‚ 775.9921875 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0$ memory_percent‚91.24689102172852 disk.rbd1.out‚ 2.0830078125 disk.sda1.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd2.out‚ 352.32421875& disk.nvme0n1.out‚43832.71435546875 disk.sda1.out‚0 proc.cpu.threads‚17 network.recv‚ 1.365706e+06" disk.nvme0c0n1.in‚ 261.28515625 disk.dm-0.in‚ 261.28515625 disk.sda3.in‚ 0.1171875 disk.sda.out‚ 775.9921875# proc.memory.rssMB‚ 2894.44921875àmöõœ…‚(:‘ ©íâÔؽÌu gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚159.512' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.902399999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0"ùõUH†‚(:½ ©íâÔ²Îw disk.dm-0.in‚ 261.28515625 disk.dm-1.out‚793.625 network.recv‚ 1.369133e+06 disk.sda.out‚793.625 disk.sda2.out‚0 disk.rbd3.in‚0" disk.dm-0.out‚43833.2783203125 disk.nvme0n1.in‚ 261.28515625 disk.sda2.in‚0% disk./.usageGB‚1065.8726119995117% disk.nvme0n1.out‚43833.2783203125 disk.sda.in‚ 0.1171875 disk.sda1.out‚0 disk.rbd0.in‚0' disk.nvme0c0n1.out‚43833.2783203125 disk.sda3.out‚793.625 disk.rbd2.out‚ 353.8984375 network.sent‚ 8.65423e+06* proc.memory.percent‚17.666316032409668 proc.cpu.threads‚17" disk.nvme0c0n1.in‚ 261.28515625 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd1.out‚ 2.0830078125# proc.memory.rssMB‚ 2894.44921875 cpu‚25.56794050077093 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375 disk.rbd2.in‚0# memory_percent‚91.2468433380127( proc.memory.availableMB‚ 1434.1171875) disk./.usagePercent‚71.53160531797693 disk.drbd1777.in‚0ž!)‘‡‚(:† ¸íâÔÖ×üu gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.876' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7752 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±®ˆMˆ‚(: ¸íâÔ—ûÐx# disk.nvme0n1.out‚43931.80859375 disk.rbd1.in‚0 disk.sda1.out‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 cpu‚25.589710687675435 disk.sda3.out‚ 807.828125 disk.dm-1.in‚ 0.1171875 disk.nvme0n1.in‚ 261.28515625 disk.rbd0.in‚0 network.recv‚ 1.373812e+06 disk.rbd0.out‚ 0.2978515625 disk.dm-0.out‚43931.80859375# proc.memory.rssMB‚ 2914.34765625* proc.memory.percent‚17.787766456604004 proc.cpu.threads‚17 disk.dm-1.out‚ 807.828125 disk.rbd2.out‚ 354.0234375 disk.drbd1777.in‚0% disk.nvme0c0n1.out‚43931.80859375 disk.rbd3.in‚0 network.sent‚ 8.660356e+06) disk./.usagePercent‚71.53159866178103% disk./.usageGB‚1065.8725128173828" disk.nvme0c0n1.in‚ 261.28515625 disk.dm-0.in‚ 261.28515625# memory_percent‚91.3691520690918 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 2.14599609375 disk.rbd2.in‚0 disk.sda.in‚ 0.1171875 disk.sda.out‚ 807.828125 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375' proc.memory.availableMB‚ 1414.078125 disk.sda1.in‚0—«O2‰- _wandbruntime‚3902  _runtime‚3902¹Æ»»>ŠÁ¦ =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.37228333950042725 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.18609558045864105 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0973280593752861 +jet/accuracy_3_of_5‚0.07790765911340714 particle/accuracy_1_of_2‚1 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.65625 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3734433650970459 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_2‚1 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3308531939983368 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6667141318321228 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.14905841648578644 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1653524786233902 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4253428876399994 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3852548599243164 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.21505455672740936 8"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.395061731338501 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.055334724485874176 particle/accuracy_4_of_6‚1 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.021661626175045967 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 particle/accuracy_5_of_6‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.33771929144859314 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_3_of_4‚1 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.11340013891458511 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 8"Purity/0FRt2SRqqt0FBt/event_purity‚0.046340212225914 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3947213590145111 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2788907587528229 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3898795545101166 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.24522793292999268 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.33306407928466797 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,jet/accuracy_4_of_5‚0.010790284723043442 particle/accuracy_4_of_5‚1 particle/accuracy_6_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1111111119389534 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4359999895095825 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.29175931215286255 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4227524399757385 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19877111911773682 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.44285711646080017 @&Purity/3FRt0SRqqt0FBdëÈ ~t/event_proportion‚0.0015120967291295528 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.13819386065006256 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26162469387054443 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3710034191608429 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1658286452293396 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +jet/accuracy_2_of_5‚0.40428534150123596 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.991713285446167 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9243295788764954 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.641666829586029 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 _step‚342 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.23333331942558289 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.37179484963417053 +jet/accuracy_2_of_3‚0.20589648187160492 particle/accuracy_2_of_4‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.17400990426540375 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3884892761707306 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.42444872856140137 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.05051413178443909 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.14959271252155304 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.32669124007225037 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.41414886713027954 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_5‚0 jet/accuracy_4_of_7‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.024921448901295662 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02074451372027397 +jet/accuracy_2_of_2‚0.09329896420240402 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 -jet/accuracy_4_of_4‚0.0038579311221837997 _runtime‚3902.266003741 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6754385828971863 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.09512334316968918 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3722890615463257 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9910787343978882 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.44845229387283325 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4493357539176941 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 jet/accuracy_1_of_7‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12931033968925476 ;"Purity/1FRt1SRqqt2FBt/event_purity‚0.011904762126505375 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.36140206456184387 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4030570685863495 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.36981165409088135 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 8"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.374701589345932 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.31706345081329346 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2720605432987213 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21541424095630646 jet/accuracy_3_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0673329159617424 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4570276439189911 7"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.37164506316185 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 epoch‚6 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 *jet/accuracy_1_of_4‚0.8583444356918335 particle/accuracy_1_of_5‚1 particle/accuracy_2_of_5‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1666666716337204 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.2125459611415863 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.29370540380477905 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.448802649974823 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1525089591741562 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.3778412640094757 *jet/accuracy_1_of_6‚0.9473684430122375 trainer/global_step‚4227 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.045079369097948074 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18606741726398468 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13742464780807495 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.36933186650276184 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16440090537071228 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.17136844992637634 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988943338394165 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2993538975715637 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.015327719040215015 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.01890210062265396 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.32978272438049316 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3830313980579376 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6754385828971863 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2777777910232544 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3515625 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809552192688 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt2SRqqt*FBt/event_purity‚0.0368238165974617 +particle/specificity‚0.9508451819419861 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 jet/accuracy_2_of_7‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9984056353569031 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9784946441650391 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 (particle/accuracy‚0.8535470366477966 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.25936147570610046 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3916666507720947 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9910423159599304 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *loss/val_total_loss‚44.003048152941716 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2954118549823761 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚0.5 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3815494477748871 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9964157938957214 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9913268089294434 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.37276849150657654 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.2479523867368698 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3767361342906952 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0021505376789718866 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4643571674823761 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3461998403072357 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3849351108074188 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.29909756779670715 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_6‚1 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9904725551605225 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9889571666717529 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27676770091056824 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 % _timestamp‚1.7883931465076225e+09 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.32869336009025574 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.32334011793136597 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.33041706681251526 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09900711476802826 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13515408337116241 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3071160614490509 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2199999988079071 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.41601380705833435 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07777778059244156 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.10383625328540802 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3499053716659546 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.41473448276519775 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16820569336414337 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4462585151195526 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1824571192264557 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.013407735154032707 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1798907369375229 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9895355105400085 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.18312758207321167 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09780790656805038 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9848148226737976 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1435672640800476 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.190476194024086 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.19419513642787933 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3913978636264801 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1111111119389534 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.07518518716096878 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17284224927425385 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.24077379703521729 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13532249629497528 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.190476194024086 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4509481191635132 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9928160905838013 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.25986602902412415 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1265089511871338 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006523106247186661 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.4035087525844574 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19131997227668762 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.22979798913002014 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_7_of_7‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.41473448276519775 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 +jet/accuracy_3_of_6‚0.10526315867900848 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.4000000059604645 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.49761852622032166 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3997334837913513 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9938474893569946 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06646621972322464 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.45000001788139343 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 particle/accuracy_5_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.30263158679008484 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34646713733673096 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.12893971800804138 8"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.441296249628067 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.379444420337677 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37681159377098083 particle/accuracy_1_of_3‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9956989288330078 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.33703702688217163 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.1753499060869217 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21734590828418732 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3714785873889923 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.991958737373352 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.39218246936798096 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.36079108715057373 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_7‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.404265433549881 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.10185185074806213 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8183577656745911 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 +jet/accuracy_1_of_1‚0.43321821093559265 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14350198209285736 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.23333333432674408 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0009216589969582856 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1869068294763565 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.1484253853559494 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00504012918099761 jet/accuracy_4_of_6‚0 particle/accuracy_1_of_6‚1 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9930030703544617 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9224615693092346 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.09541063010692596 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3283127546310425 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.45624998211860657 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24380917847156525 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4298049211502075 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13411085307598114 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.11340013891458511 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_6_of_7‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.39773625135421753 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4211442470550537 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.39290016889572144 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.18696346879005432 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.33535853028297424 *jet/accuracy_1_of_2‚0.6797638535499573 particle/accuracy_3_of_3‚1 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6470588445663452 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_1‚1 6 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.111115001142025 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.04545804113149643 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3650793731212616 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.11656466126441956 6 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.448151558637619 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.3359375 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.14102564752101898 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9811885356903076 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 *jet/accuracy_1_of_3‚0.8071756958961487 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809552192688 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13719463348388672 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9937912821769714 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.39531388878822327 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 +particle/sensitivity‚0.3278360664844513 'particle/f_score‚0.4114106595516205 jet/accuracy_5_of_6‚0 particle/accuracy_3_of_7‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3892895579338074 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6785714030265808 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01077850442379713 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1820022314786911 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 6validation_average_jet_accuracy‚0.3913150131702423 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4785449802875519 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.393942266702652 particle/accuracy_3_of_6‚1̺¸è‹Ʀ epoch‚6 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 *jet/accuracy_1_of_4‚0.8583444356918335 particle/accuracy_1_of_5‚1 particle/accuracy_2_of_5‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1666666716337204 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.2125459611415863 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.29370540380477905 ,&Purity/2FRt3SRqqt2F&VF¹ùBt/event_proportion‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.448802649974823 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1525089591741562 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.3778412640094757 *jet/accuracy_1_of_6‚0.9473684430122375 trainer/global_step‚4227 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.045079369097948074 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18606741726398468 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13742464780807495 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.36933186650276184 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16440090537071228 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.17136844992637634 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988943338394165 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2993538975715637 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.015327719040215015 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.01890210062265396 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.32978272438049316 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3830313980579376 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6754385828971863 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2777777910232544 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3515625 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809552192688 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt2SRqqt*FBt/event_purity‚0.0368238165974617 +particle/specificity‚0.9508451819419861 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 jet/accuracy_2_of_7‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9984056353569031 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9784946441650391 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 (particle/accuracy‚0.8535470366477966 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.25936147570610046 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3916666507720947 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9910423159599304 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *loss/val_total_loss‚44.003048152941716 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2954118549823761 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 particle/accuracy_3_of_5‚1 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚0.5 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3815494477748871 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9964157938957214 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9913268089294434 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.37276849150657654 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.2479523867368698 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3767361342906952 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0021505376789718866 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4643571674823761 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3461998403072357 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3849351108074188 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.29909756779670715 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_6‚1 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9904725551605225 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9889571666717529 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27676770091056824 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 % _timestamp‚1.7883931465076225e+09 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.32869336009025574 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.32334011793136597 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.33041706681251526 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09900711476802826 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13515408337116241 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3071160614490509 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2199999988079071 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.41601380705833435 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07777778059244156 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.10383625328540802 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3499053716659546 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.41473448276519775 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16820569336414337 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4462585151195526 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1824571192264557 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.013407735154032707 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1798907369375229 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9895355105400085 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.18312758207321167 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09780790656805038 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9848148226737976 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1435672640800476 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.190476194024086 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.19419513642787933 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3913978636264801 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1111111119389534 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.07518518716096878 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17284224927425385 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.24077379703521729 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13532249629497528 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.190476194024086 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.4509481191635132 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9928160905838013 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.25986602902412415 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1265089511871338 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006523106247186661 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_6‚0.4035087525844574 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19131997227668762 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.22979798913002014 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_7_of_7‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.41473448276519775 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 +jet/accuracy_3_of_6‚0.10526315867900848 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.4000000059604645 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.49761852622032166 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3997334837913513 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9938474893569946 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06646621972322464 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.45000001788139343 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 particle/accuracy_5_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.30263158679008484 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34646713733673096 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.12893971800804138 8"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.441296249628067 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 6 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.379444420337677 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37681159377098083 particle/accuracy_1_of_3‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9956989288330078 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.33703702688217163 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.1753499060869217 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21734590828418732 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3714785873889923 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.991958737373352 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.39218246936798096 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.36079108715057373 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_7‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.404265433549881 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.10185185074806213 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8183577656745911 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 +jet/accuracy_1_of_1‚0.43321821093559265 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14350198209285736 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.23333333432674408 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0009216589969582856 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.1869068294763565 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.1484253853559494 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt*SRqqt*FBt/event_purity‚0.00504012918099761 jet/accuracy_4_of_6‚0 particle/accuracy_1_of_6‚1 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9930030703544617 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9224615693092346 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.09541063010692596 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3283127546310425 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.45624998211860657 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24380917847156525 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4298049211502075 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13411085307598114 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.11340013891458511 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_6_of_7‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.39773625135421753 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4211442470550537 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.39290016889572144 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.18696346879005432 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.33535853028297424 *jet/accuracy_1_of_2‚0.6797638535499573 particle/accuracy_3_of_3‚1 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6470588445663452 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_1‚1 6 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.111115001142025 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.04545804113149643 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3650793731212616 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.11656466126441956 6 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.448151558637619 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.3359375 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.14102564752101898 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9811885356903076 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 *jet/accuracy_1_of_3‚0.8071756958961487 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809552192688 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13719463348388672 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9937912821769714 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.39531388878822327 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 +particle/sensitivity‚0.3278360664844513 'particle/f_score‚0.4114106595516205 jet/accuracy_5_of_6‚0 particle/accuracy_3_of_7‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3892895579338074 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6785714030265808 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01077850442379713 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1820022314786911 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 6validation_average_jet_accuracy‚0.3913150131702423 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4785449802875519 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.393942266702652 particle/accuracy_3_of_6‚1 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.37228333950042725 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.18609558045864105 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.0973280593752861 +jet/accuracy_3_of_5‚0.07790765911340714 particle/accuracy_1_of_2‚1 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.65625 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3734433650970459 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_2‚1 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3308531939983368 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6667141318321228 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.14905841648578644 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1653524786233902 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4253428876399994 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3852548599243164 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.21505455672740936 8"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.395061731338501 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.055334724485874176 particle/accuracy_4_of_6‚1 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.021661626175045967 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 particle/accuracy_5_of_6‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.33771929144859314 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_3_of_4‚1 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.11340013891458511 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 8"Purity/0FRt2SRqqt0FBt/event_purity‚0.046340212225914 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3947213590145111 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2788907587528229 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3898795545101166 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.24522793292999268 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.33306407928466797 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,jet/accuracy_4_of_5‚0.010790284723043442 particle/accuracy_4_of_5‚1 particle/accuracy_6_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1111111119389534 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4359999895095825 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.29175931215286255 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4227524399757385 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19877111911773682 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.44285711646080017 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.13819386065006256 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26162469387054443 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3710034191608429 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1658286452293396 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +jet/accuracy_2_of_5‚0.40428534150123596 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.991713285446167 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.003225806402042508 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9243295788764954 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.641666829586029 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 _step‚342 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.23333331942558289 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.37179484963417053 +jet/accuracy_2_of_3‚0.20589648187160492 particle/accuracy_2_of_4‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.17400990426540375 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3884892761707306 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.42444872856140137 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.05051413178443909 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.14959271252155304 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/evüA¤_lent_proportion‚0.00025201612152159214 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.32669124007225037 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.41414886713027954 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_5‚0 jet/accuracy_4_of_7‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.024921448901295662 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02074451372027397 +jet/accuracy_2_of_2‚0.09329896420240402 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 -jet/accuracy_4_of_4‚0.0038579311221837997 _runtime‚3902.266003741 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6754385828971863 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.09512334316968918 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3722890615463257 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9910787343978882 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.44845229387283325 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4493357539176941 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 jet/accuracy_1_of_7‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12931033968925476 ;"Purity/1FRt1SRqqt2FBt/event_purity‚0.011904762126505375 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.36140206456184387 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4030570685863495 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.36981165409088135 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 8"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.374701589345932 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.31706345081329346 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2720605432987213 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21541424095630646 jet/accuracy_3_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0673329159617424 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4570276439189911 7"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.37164506316185 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607Öl¹%¯‘Œ‚(:† ÇíâÔ½Êu gpu.0.gpu‚29.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚149.145' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.829 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0?¾Ì°Z‚(:Ï ÇíâÔêÔÊx disk.dm-0.in‚ 261.28515625# proc.memory.rssMB‚ 2894.44921875 network.recv‚ 1.378663e+06 disk.rbd2.out‚ 354.04296875 disk.dm-1.out‚ 823.2109375( proc.memory.availableMB‚ 1435.5859375 cpu‚25.59348175545706 disk.rbd1.out‚ 2.14599609375 disk.rbd0.in‚0( disk.nvme0c0n1.out‚44174.99072265625 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 823.2109375* proc.memory.percent‚17.666316032409668 network.sent‚ 8.666856e+06$ disk./.usageGB‚1065.872901916504 disk.rbd2.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0# disk.dm-0.out‚44174.99072265625$ memory_percent‚91.23787879943848 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0 proc.cpu.threads‚17 disk.rbd1.in‚0& disk.nvme0n1.out‚44174.99072265625 disk.sda1.out‚0 disk.sda2.in‚0 disk.dm-1.in‚ 0.1171875) disk./.usagePercent‚71.53162477454953 disk.nvme0n1.in‚ 261.28515625 disk.sda.in‚ 0.1171875 disk.sda3.in‚ 0.1171875 disk.sda.out‚ 823.2109375" disk.nvme0c0n1.in‚ 261.28515625 disk.rbd3.in‚0C·È2Ž- _wandbruntime‚3923  _runtime‚3923bº3È ' lr-AdamW/pg1‚0.00042064818792224534 ' lr-AdamW/pg2‚0.00042064818792224534 trainer/global_step‚4249 $ _timestamp‚1.788393167622472e+09 _runtime‚3923.378421392 _step‚343ú„¯³ÍÇ ' lr-AdamW/pg2‚0.00042064818792224534 trainer/global_step‚4249 $ _timestamp‚1.788393167622472e+09 _runtime‚3923.378421392 _step‚343 ' lr-AdamW/pg1‚0.00042064818792224534×Ûr…ë2‘- _wandbruntime‚3923  _runtime‚3923«üÞfç'’áO LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3488022546862 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚279487184144384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚90705927 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚67593866 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚90728525 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚514 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚44737944 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚286464535548928 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 163712505 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚3923.379701765 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚44737947 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚292456439240704 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3507766792704 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3488214543402 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚67593349 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚517 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚289972089123328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚289972089123328 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚282975415727146 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788393167623493e+09 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3683246444032 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚72983978 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚286464518509568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚288773451437568 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 163689905 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚90705927 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚289972302341632 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚22855919 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3507766792704 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚72983978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3683012706816 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 4943455530Íãˆì'“æO VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚282975206691246 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚289972089123328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚289972089123328 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚282975415727146 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788393167623493e+09 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3683246444032 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚72983978 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚286464518509568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚288773451437568 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 163689905 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚90705927 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚289972302341632 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚22855919 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3507766792704 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚72983978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3683012706816 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3488022546862 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚279487184144384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚90705927 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚67593866 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚90728525 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚514 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚44737944 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚286464535548928 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 163712505 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚3923.379701765 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚44737947 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚292456439240704 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3507766792704 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3488214543402 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚67593349 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚517 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 J6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚73000498 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3508369004544 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚283039808994878 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚44747981 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3508565196800 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22860378 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚290037926551040 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3508565196800 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚90749423 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚292522820012032 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚283039599958770 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚67608356 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚288838976933888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚73000498 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 163749923 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚44747978 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚345 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ Ô§h(“11322690560 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 163727297 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚90726799 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 163727297 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3488816429810 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 $ _timestamp‚1.788393168388942e+09 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 163749923 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚4249 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚506 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚290038139782656 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚90726799 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243ôŠ×ö&–ðM >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚290037926551040 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3508565196800 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚90749423 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚292522820012032 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚283039599958770 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚67608356 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚288838976933888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚73000498 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 163749923 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚44747978 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _step‚345 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 163727297 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚90726799 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 163727297 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3488816429810 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 $ _timestamp‚1.788393168388942e+09 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 163749923 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚4249 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚506 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚290038139782656 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚90726799 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚73000498 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3508369004544 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚283039808994878 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚44747981 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3508565196800 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚22860378 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚73809980 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚91775477 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3725302785536 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚282667153373184 _step‚348 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚293263988337152 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 165562857 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚68353596 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3527935862810 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚91752879 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚4299 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23113438 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3547517981184 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚289716487395328 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 $ _timestamp‚1.788393210291644e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 165585457 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚286195106275354 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3547517981184 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚514 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚282667170412544 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚293264201555456 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3547714160128 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚73809980 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚91775477 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3725302785536 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚282667153373184 _step‚348 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚293263988337152 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 165562857 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚68353596 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3527935862810 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚91752879 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚4299 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23113438 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3547517981184 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚289716487395328 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 $ _timestamp‚1.788393210291644e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 165585457 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚286195106275354 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3547517981184 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚514 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚282667170412544 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚293264201555456 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3547714160128 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚91773751 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚68368814 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚45249681 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚45249678 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 $ _timestamp‚1.788393211049465e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚289781509392896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3548512564224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚91796375 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚292115251749888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚293329825764864 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚293330038996480 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚289781526432256 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚292115226846208 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3726150818816 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚91796375 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚295841632098304 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚289781526432256 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚293330038996480 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚499 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23119133 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3548316371968 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚286259499543086 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚282730769797120 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 165622875 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23118637 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 165622875 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 _runtime‚3966.806150229 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚73826498 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3528729745966 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚68368315 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 trainer/global_step‚4299 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚73826498 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚73826500 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243´ÛÛ&ö&¨ðM WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚293330038996480 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚499 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23119133 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3548316371968 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚286259499543086 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚282730769797120 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 165622875 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23118637 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 165622875 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 _runtime‚3966.806150229 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚73826498 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3528729745966 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚68368315 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 trainer/global_step‚4299 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚73826498 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚73826500 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3548316371968 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 165600249 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚91773751 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚68368814 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚45249681 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚45249678 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 $ _timestamp‚1.788393211049465e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚289781509392896 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3548512564224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚91796375 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚292115251749888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚293329825764864 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚293330038996480 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚289781526432256 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚292115226846208 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3726150818816 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚91796375 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚295841632098304 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚289781526432256Ý,˜¦2©- _wandbruntime‚3966  _runtime‚3966õ³ø\ªÖ /loss/FRt2/detection_loss‚1.4880328306227648 1loss/SRqqt4/detection_loss‚2.4122390194215635 0loss/FRt3/assignment_loss‚3.3157822828035095 1loss/FBt2/assignment_loss‚0.09232846705547128 1loss/FBt4/assignment_loss‚0.03728397348563054 .loss/FRt4/detection_loss‚2.360981055668422 trainer/global_step‚4299 2loss/SRqqt2/assignment_loss‚0.9898708173926888 .loss/FBt4/detection_loss‚2.685113557747432 0loss/FBt1/assignment_loss‚0.0796829376840833 2loss/FBt3/assignment_loss‚0.041244342095244343 1loss/SRqqt2/detection_loss‚0.9851242295082878 1loss/SRqqt3/detection_loss‚1.9718820769772973 _step‚350 2loss/SRqqt1/assignment_loss‚1.5777681367091683 /loss/FBt2/detection_loss‚0.4506275623219174 epoch‚7 /loss/FRt4/assignment_loss‚3.600442884933381 .loss/FBt1/detection_loss‚2.823738020658493 /loss/FBt3/detection_loss‚3.3993151829792905 _runtime‚ 3966.8073603 2loss/SRqqt3/assignment_loss‚1.5408529509292093 2loss/SRqqt4/assignment_loss‚1.6291095836870912 1loss/SRqqt1/detection_loss‚1.8827681752796075 % _timestamp‚1.7883932110515213e+09 /loss/FRt1/assignment_loss‚3.069967451881855 0loss/FRt2/assignment_loss‚1.4556387307447032 .loss/FRt1/detection_loss‚2.324903422213615 /loss/FRt3/detection_loss‚2.6344611697100304 &loss/total_loss‚42.849158862510755<žÃèa«Û 2loss/SRqqt1/assignment_loss‚1.5777681367091683 /loss/FBt2/detection_loss‚0.4506275623219174 epoch‚7 /loss/FRt4/assignment_loss‚3.600442884933381 .loss/FBt1/detection_loss‚2.823738020658493 /loss/FBt3/detection_loss‚3.3993151829792905 _runtime‚ 3966.8073603 2loss/SRqqt3/assignment_loss‚1.5408529509292093 2loss/SRqqt4/assignment_loss‚1.6291095836870912 1loss/SRqqt1/detection_loss‚1.8827681752796075 % _timestamp‚1.7883932110515213e+09 /loss/FRt1/assignment_loss‚3.069967451881855 0loss/FRt2/assignment_loss‚1.4556387307447032 .loss/FRt1/detection_loss‚2.324903422213615 /loss/FRt3/detection_loss‚2.6344611697100304 &loss/total_loss‚42.849158862510755 /loss/FRt2/detection_loss‚1.4880328306227648 1loss/SRqqt4/detection_loss‚2.4122390194215635 0loss/FRt3/assignment_loss‚3.3157822828035095 1loss/FBt2/assignment_loss‚0.09232846705547128 1loss/FBt4/assignment_loss‚0.03728397348563054 .loss/FRt4/detection_loss‚2.360981055668422 trainer/global_step‚4299 2loss/SRqqt2/assignment_loss‚0.9898708173926888 .loss/FBt4/detection_loss‚2.685113557747432 0loss/FBt1/assignment_loss‚0.0796829376840833 2loss/FBt3/assignment_loss‚0.041244342095244343 1loss/SRqqt2/detection_loss‚0.9851242295082878 1loss/SRqqt3/detection_loss‚1.9718820769772973 _step‚350ÞhbŸ¸8¬‚(:‡ ƒîâÔ©Þµv gpu.0.gpu‚27.0 gpu.0.me¾"Zmory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚102.782' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.5564 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶Ä’*[­‚(:Ð ƒîâÔ•›Ïx disk.dm-0.in‚ 261.28515625 disk.rbd2.out‚ 356.1328125 disk.sda.out‚ 854.52734375" disk.nvme0c0n1.in‚ 261.28515625 disk.dm-1.out‚ 854.52734375 disk.nvme0n1.in‚ 261.28515625 disk.rbd0.in‚0 cpu‚25.594373822403522) disk./.usagePercent‚71.53156947692213 disk.sda2.out‚0 network.recv‚ 1.399403e+06 disk.rbd3.in‚0 disk.drbd1777.in‚0$ memory_percent‚91.23849868774414 disk.rbd1.out‚ 2.208984375 disk.drbd1777.out‚0* proc.memory.percent‚17.666316032409668 network.sent‚ 8.868787e+06 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.rbd0.out‚ 0.2978515625# proc.memory.rssMB‚ 2894.44921875# disk.dm-0.out‚44515.95654296875 disk.sda3.out‚ 854.52734375 disk.sda.in‚ 0.1171875 disk.sda1.out‚0' proc.memory.availableMB‚ 1435.484375% disk./.usageGB‚1065.8720779418945 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875& disk.nvme0n1.out‚44515.95654296875 disk.sda1.in‚0( disk.nvme0c0n1.out‚44515.95654296875 disk.sda2.in‚0 proc.cpu.threads‚17 disk.rbd3.out‚ 2.349609375’—‘®‚(:† ’îâÔíÑúv gpu.0.gpu‚56.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚93.069' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6138 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0áújgX¯‚(:Í ’îâÔÿÞÑw disk.sda2.out‚0 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 872.33203125 disk.rbd1.in‚0 disk.sda.out‚ 872.33203125 disk.sda1.out‚0& disk.nvme0c0n1.out‚44516.373046875 disk.rbd3.out‚ 2.349609375! disk.dm-0.out‚44516.373046875 disk.dm-1.in‚ 0.1171875$ memory_percent‚91.23880863189697) proc.memory.availableMB‚ 1435.43359375* proc.memory.percent‚17.666316032409668 disk.sda1.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 2894.44921875% disk./.usageGB‚1065.8721466064453 disk.sda.in‚ 0.1171875 disk.drbd1777.in‚0 disk.dm-0.in‚ 261.28515625 disk.rbd2.out‚ 357.64453125 cpu‚25.594651410314228 disk.drbd1777.out‚0 disk.rbd3.in‚0 proc.cpu.threads‚17 network.sent‚ 8.955447e+06 network.recv‚ 1.404561e+06 disk.nvme0n1.in‚ 261.28515625 disk.rbd0.out‚ 0.2978515625 disk.dm-1.out‚ 872.33203125 disk.rbd2.in‚0 disk.rbd1.out‚ 2.208984375) disk./.usagePercent‚71.53157408505774$ disk.nvme0n1.out‚44516.373046875" disk.nvme0c0n1.in‚ 261.28515625(ûvž°‚(:’ ¡îâÔÖÅÕu gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.412' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.282400000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒ$5ê^±‚(:Ó ¡îâÔ¤¹êw disk.dm-1.out‚ 889.01953125& disk.nvme0n1.out‚44615.22607421875 disk.sda.out‚ 889.01953125" disk.nvme0c0n1.in‚ 261.28515625$ disk./.usageGB‚1065.872127532959 disk.rbd1.out‚ 2.208984375 disk.nvme0n1.in‚ 261.28515625) proc.memory.availableMB‚ 1435.41796875 disk.rbd1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.dm-0.in‚ 261.28515625# disk.dm-0.out‚44615.22607421875$ memory_percent‚91.23890399932861# proc.memory.rssMB‚ 2894.44921875* proc.memory.percent‚17.666316032409668 network.sent‚ 8.96086e+06 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚44615.22607421875 disk.rbd3.out‚ 2.349609375 disk.sda3.in‚ 0.1171875 disk.rbd0.out‚ 0.2978515625 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 889.01953125 cpu‚25.59472935532752 network.recv‚ 1.408135e+06) disk./.usagePercent‚71.53157280502008 disk.rbd0.in‚0 disk.rbd3.in‚0! disk.rbd2.out‚357.80517578125 disk.sda.in‚ 0.1171875 proc.cpu.threads‚17CC”r2²- _wandbruntime‚4008  _runtime‚4008Ì"Lɳà _step‚351 ' lr-AdamW/pg1‚0.00037870133992589863 ' lr-AdamW/pg2‚0.00037870133992589863 trainer/global_step‚4349 % _timestamp‚1.7883932530476782e+09 _runtime‚4008.8037140194˜¼µÎ´È _step‚351 ' lr-AdamW/pg1‚0.00037870133992589863 ' lr-AdamW/pg2‚0.00037870133992589863 trainer/global_step‚4349 % _timestamp‚1.7883932530476782e+09 _runtime‚4008.803714019߸‚þh2µ- _wandbruntime‚4008  _runtime‚4008ð)¼pè'¶âO LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚74635978 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3767427144192 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚292968439241728 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚296555887550976 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3587465348608 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚295325976165888 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3587465348608 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚74635980 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚299093668242432 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚4349 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚285847122601984 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚74635978 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚92799831 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚296556100769280 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚299093403310080 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚92822429 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23372311 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚292968439241728 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚292968422202368 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883932530488486e+09 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚45741344 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3587661527552 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚289414796823562 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540¸ÜƒØí'·çO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚74635980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚296555887550976 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚296556100769280 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚299093403310080 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚92822429 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23372311 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚292968439241728 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚292968422202368 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883932530488486e+09 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚45741344 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3587661527552 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚289414796823562 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚74635978 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3767427144192 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚292968439241728 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚296555887550976 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3587465348608 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚295325976165888 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3587465348608 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚74635980 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚299093668242432 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 trainer/global_step‚4349 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚285847122601984 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚74635978 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚92799831 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚69128829 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚92820703 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚92820703 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3768505923584 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3568259068626 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚293033461239296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚45751378 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3768257519616 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚353 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚299160032489472 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚289479190091294 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3588459931648 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3588263739392 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚285910739025920 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚285910721986560 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚293033478278656 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚92843327 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚299159759181824 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚536 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚74652498 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚293033461239296 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚74652500 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7883932538077064e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚74652500 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚296621938210304 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚296621938210304 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 167495827 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23377451 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750ö2Çu÷&ºñM D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 167473201 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3588263739392 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚74652498 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 167473201 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3568451065374 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3588459931648 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚69129365 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23377984 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚69128829 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚92820703 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚92820703 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3768505923584 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3568259068626 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚293033461239296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚45751378 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3768257519616 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚353 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚299160032489472 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚289479190091294 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3588459931648 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3588263739392 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚285910739025920 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚285910721986560 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚293033478278656 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚92843327 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚299159759181824 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚536 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚74652498 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚293033461239296 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚74652500 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7883932538077064e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚74652500 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚296621938210304 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚296621938210304 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 167495827 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23377451 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750áž KN2»- _wandbruntime‚4009  _runtime‚4009èƒÊ$Y¼Ó .loss/FBt1/detection_loss‚2.264207753870222 epoch‚7 % _timestamp‚1.7883932538093426e+09 trainer/global_step‚4349 0loss/FBt4/assignment_loss‚0.6138617407699485 1loss/SRqqt4/detection_loss‚2.2405703961849213 /loss/FBt2/detection_loss‚0.6742672190613905 _runtime‚4009.565188529 2loss/SRqqt3/assignment_loss‚1.8500700476978507 1loss/SRqqt1/detection_loss‚1.7381737226933505 0loss/FRt4/assignment_loss‚3.5380649549119614 /loss/FRt3/detection_loss‚2.5625383888566216 .loss/FBt3/detection_loss‚2.860852447423068 1loss/SRqqt4/assignment_loss‚1.806108904542861 /loss/FRt2/detection_loss‚1.6443797933445736 _step‚354 /loss/FRt3/assignment_loss‚3.502708066341489 1loss/FBt2/assignment_loss‚0.11641983618437451 0loss/FBt3/assignment_loss‚0.4819363881635416 /loss/FRt4/detection_loss‚2.4050657490597054 0loss/SRqqt3/detection_loss‚1.844668951062929 0loss/FRt2/assignment_loss‚1.5043449521064758 2loss/SRqqt1/assignment_loss‚1.4203719005186473 2loss/SRqqt2/assignment_loss‚0.9720731573628968 .loss/FRt1/detection_loss‚2.185587886501761 .loss/FBt4/detection_loss‚2.317145163193345 %loss/total_loss‚42.57988293697673 /loss/FRt1/assignment_loss‚3.055604474217284 1loss/FBt1/assignment_loss‚0.12842751975716105 1loss/SRqqt2/detection_loss‚0.8524335231503536ŽÐ‡^½Ø .loss/FBt4/detection_loss‚2.317145163193345 %loss/total_loss‚42.57988293697673 /loss/FRt1/assignment_loss‚3.055604474217284 1loss/FBt1/assignment_loss‚0.12842751975716105 1loss/SRqqt2/detection_loss‚0.8524335231503536 .loss/FBt1/detection_loss‚2.264207753870222 epoch‚7 % _timestamp‚1.7883932538093426e+09 trainer/global_step‚4349 0loss/FBt4/assignment_loss‚0.6138617407699485 1loss/SRqqt4/detection_loss‚2.2405703961849213 /loss/FBt2/detection_loss‚0.6742672190613905 _runtime‚4009.565188529 2loss/SRqqt3/assignment_loss‚1.8500700476978507 1loss/SRqqt1/detection_loss‚1.7381737226933505 0loss/FRt4/assignment_loss‚3.5380649549119614 /loss/FRt3/detection_loss‚2.5625383888566216 .loss/FBt3/detection_loss‚2.860852447423068 1loss/SRqqt4/assignment_loss‚1.806108904542861 /loss/FRt2/detection_loss‚1.6443797933445736 _step‚354 /loss/FRt3/assignment_loss‚3.502708066341489 1loss/FBt2/assignment_loss‚0.11641983618437451 0loss/FBt3/assignment_loss‚0.4819363881635416 /loss/FRt4/detection_loss‚2.4050657490597054 0loss/SRqqt3/detection_loss‚1.844668951062929 0loss/FRt2/assignment_loss‚1.5043449521064758 2loss/SRqqt1/assignment_loss‚1.4203719005186473 2loss/SRqqt2/assignment_loss‚0.9720731573628968 .loss/FRt1/detection_loss‚2.185587886501761âÿ¿ý’¾‚(:‡ °îâÔåþÑu gpu.0.gpu‚21.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.673' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.9346 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÌŽT¿‚(:É °îâÔ½ÛÙx! proc.memory.rssMB‚ 2894.453125* proc.memory.percent‚17.666339874267578 disk.rbd3.out‚ 2.349609375 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 906.171875 disk.dm-1.out‚ 906.171875 disk.rbd2.in‚0& disk.nvme0n1.out‚44713.59716796875 proc.cpu.threads‚17 disk.sda2.in‚0 disk.drbd1777.out‚0 cpu‚25.594919135561668" disk.nvme0c0n1.in‚ 261.28515625# disk.dm-0.out‚44713.59716796875 disk.sda.in‚ 0.1171875 disk.rbd0.in‚0 disk.rbd1.out‚ 2.208984375 disk.sda.out‚ 906.171875 disk.sda1.in‚0( disk.nvme0c0n1.out‚44713.59716796875 disk.rbd3.in‚0 disk.nvme0n1.in‚ 261.28515625 disk.sda1.out‚0 network.sent‚ 8.966984e+06) disk./.usagePercent‚71.53157152498241% disk./.usageGB‚1065.8721084594727 disk.dm-0.in‚ 261.28515625 network.recv‚ 1.41257e+06 disk.sda2.out‚0$ memory_percent‚91.25175476074219 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0! disk.rbd2.out‚359.31689453125 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0% proc.memory.availableMB‚ 1433.3125Õ/úœÀ‚(:‘ ¿îâÔÚ…Öu gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.36' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.872000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0…õªšQÁ‚(:Æ ¿îâÔ‘–·x proc.cpu.threads‚17 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.dm-0.out‚ 44714.0390625 disk.sda3.out‚ 923.41796875# memory_percent‚91.2513017654419* proc.memory.percent‚17.666339874267578 network.sent‚ 9.054814e+06" disk.nvme0n1.out‚ 44714.0390625! disk.rbd2.out‚359.42236328125 network.recv‚ 1.418931e+06$ disk./.usageGB‚1065.872127532959 disk.sda1.out‚0" disk.nvme0c0n1.in‚ 261.28515625$ disk.nvme0c0n1.out‚ 44714.0390625 disk.rbd3.in‚0 cpu‚25.595061711941536 disk.sda.out‚ 923.41796875 disk.dm-1.out‚ 923.41796875) proc.memory.availableMB‚ 1433.38671875) disk./.usagePercent‚71.53157280502008 disk.nvme0n1.in‚ 261.28515625 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.rbd2.in‚0 disk.sda1.in‚0 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.28515625! proc.memory.rssMB‚ 2894.453125 disk.rbd0.in‚0 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.sda.in‚ 0.1171875 disk.rbd0.out‚ 0.2978515625 disk.rbd1.out‚ 2.208984375¶ü}‚(:’ ÎîâÔ®»µu gpu.0.gpu‚32.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.383' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.076600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0vZ¼bZÂ(:Ï ÎîâÔà× x disk.sda1.out‚0 disk.drbd1777.out‚0 disk.sda2.in‚0) proc.memory.availableMB‚ 1433.29296875# proc.memory.rssMB‚ 2894.45703125 network.sent‚ 9.060109e+06% disk./.usageGB‚1065.8721961975098 disk.rbd1.out‚ 2.208984375 disk.sda1.in‚0 proc.cpu.threads‚17) disk./.usagePercent‚71.53157741315569 disk.rbd3.in‚0! disk.rbd2.out‚360.93408203125 disk.sda2.out‚0$ memory_percent‚91.25187397003174' disk.nvme0c0n1.out‚44812.3857421875 disk.sda3.out‚ 924.6796875 disk.rbd2.in‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 924.6796875 disk.sda.out‚ 924.6796875" disk.nvme0c0n1.in‚ 261.28515625) proc.memory.percent‚17.66636371612549 disk.rbd3.out‚ 2.349609375% disk.nvme0n1.out‚44812.3857421875 disk.sda.in‚ 0.1171875 disk.rbd0.out‚ 0.2978515625 network.recv‚ 1.422697e+06 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.28515625 disk.rbd0.in‚0 disk.drbd1777.in‚0 cpu‚25.595262335874022 disk.dm-0.in‚ 261.28515625" disk.dm-0.out‚44812.3857421875 disk.sda3.in‚ 0.1171875j’Oë2Ä- _wandbruntime‚4051  _runtime‚4051jù¸’ÅÅ¿ _runtime‚4051.407911924 _step‚355 % lr-AdamW/pg1‚0.000358321135248169 % lr-AdamW/pg2‚0.000358321135248169 trainer/global_step‚4399 % _timestamp‚1.7883932956516852e+09;‚<7ÊÆÄ _runtime‚4051.407911924 _step‚355 % lr-AdamW/pg1‚0.000358321135248169 % lr-AdamW/pg2‚0.000358321135248169 trainer/global_step‚4399 % _timestamp‚1.7883932956516852e+09ãåÖ¡2Ç- _wandbruntime‚4051  _runtime‚4051t'./ç'ÈáO JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚296220391088128 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚93869381 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚302412160867840 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚4399 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚298602275885568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚299847999983104 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚46243047 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚23633240 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 169308761 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚296220374048768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3627608894976 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚356 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚93869381 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚46243044 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23632735 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚302411902226944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚75461980 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3627412716032 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 _runtime‚4051.408928981 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚296220391088128 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚75461978 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚292634278335870 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚505 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚299847786764800 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3627412716032 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚299847999983104 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚93846783 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚93846783 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚299847786764800 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚75461978 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚292634487371770 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3809884982272 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3607186505086 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚298602250981888 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 $ _timestamp‚1.788393295652527e+098JוÉæO ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3607186505086 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚298602250981888 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 $ _timestamp‚1.788393295652527e+09 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚296220391088128 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚93869381 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚302412160867840 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 trainer/global_step‚4399 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚298602275885568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚299847999983104 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚46243047 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚23633240 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 169308761 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚296220374048768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3627608894976 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚356 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚93869381 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚46243044 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23632735 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚302411902226944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚75461980 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3627412716032 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 _runtime‚4051.408928981 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚296220391088128 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚75461978 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚292634278335870 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚505 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚299847786764800 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3627412716032 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚299847999983104 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚93846783 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚93846783 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚299847786764800 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚75461978 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚292634487371770 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3809884982272 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896äì :é2Ê- _wandbruntime‚4052  _runtime‚4052É}¼8ò&ËìM YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚289090691215360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 169368779 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚292698671603394 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚75478498 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚302478265260544 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚299913837424128 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3628407299072 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3608172384782 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3607980388034 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3628407299072 _step‚357 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3628211106816 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚69891529 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚298667801381888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚299913837424128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚532 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚46253078 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚46253081 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚93890279 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚69890994 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23638448 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚4399 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚93867655 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23637916 % _timestamp‚1.7883932964350524e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚296285430125056 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚296285430125056 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚93890279 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚4052.191698336 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3628211106816 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚298667776478208 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚296285413085696 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 169346153 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚75478500 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 169368779 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚93867655 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚296285413085696 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 169346153 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚302478528082432 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚292698880639502 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚75478500 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3810488782336 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0÷ïr»÷&ÌñM J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚75478498 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3810726700544 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚292698671603394 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚75478498 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚302478265260544 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚299913837424128 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3628407299072 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3608172384782 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3607980388034 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3628407299072 _step‚357 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3628211106816 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚69891529 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚298667801381888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚299913837424128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚532 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚46253078 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚46253081 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚93890279 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚69890994 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23638448 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚4399 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚93867655 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23637916 % _timestamp‚1.7883932964350524e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚296285430125056 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚296285430125056 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚93890279 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚4052.191698336 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3628211106816 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚298667776478208 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚296285413085696 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 169346153 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚75478500 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 169368779 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚93867655 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚296285413085696 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 169346153 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚302478528082432 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚292698880639502 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚75478500 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3810488782336 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚289090691215360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 169368779 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 EDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _runtime‚4094.417555146 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚94916333 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 _step‚360 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚303139685978624 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚305730270581760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚295853968884078 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚94916333 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚299472325895168 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚23894364 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚303139685978624 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23893851 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚301878525797888 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚299472342934528 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚299472325895168 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚303139899196928 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3647099821034 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 trainer/global_step‚4449 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3667556262400 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚76287980 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚94893735 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚516 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3851986909696 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560ñØ ¿ì'ÙæO L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚94893735 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚516 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3851986909696 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚305730537611264 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚295854177919978 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚301878550701568 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3851744783872 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 $ _timestamp‚1.788393338661304e+09 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚76287980 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3667360083456 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3646907824494 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 171204313 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 _runtime‚4094.417555146 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚94916333 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 _step‚360 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚303139685978624 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚305730270581760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚295853968884078 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚94916333 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚299472325895168 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚23894364 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚303139685978624 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚23893851 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚301878525797888 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚299472342934528 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚299472325895168 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚303139899196928 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3647099821034 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 trainer/global_step‚4449 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3667556262400 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚76287980 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0èÅ%˜ø‘Ú‚(:† ûîâÔù¹òu gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚133.175' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.635 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ðò½PMÛ‚(: ûîâÔ©‰ëx& disk.nvme0c0n1.out‚45010.931640625# proc.memory.rssMB‚ 2894.45703125 network.recv‚ 1.436617e+06) disk./.usagePercent‚71.53177197888171% disk./.usageGB‚1065.8750953674316 disk.nvme0n1.in‚ 261.28515625 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.sda.out‚971.1875 disk.sda2.in‚0 disk.sda3.in‚ 0.1171875 disk.sda3.out‚971.1875 disk.rbd1.out‚ 2.208984375 disk.sda.in‚ 0.1171875" disk.nvme0c0n1.in‚ 261.28515625 disk.rbd3.out‚ 2.349609375$ memory_percent‚92.12706089019775 network.sent‚ 9.158659e+06 disk.rbd2.out‚364.4384765625! disk.dm-0.out‚45010.931640625 proc.cpu.threads‚17 disk.dm-1.in‚ 0.1171875 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.dm-0.in‚ 261.28515625) proc.memory.percent‚17.66636371612549 cpu‚25.595781990780377 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0 disk.dm-1.out‚971.1875$ disk.nvme0n1.out‚45010.931640625 disk.sda1.out‚0 disk.drbd1777.out‚0) proc.memory.availableMB‚ 1289.90234375ˆN)2Ü- _wandbruntime‚4095  _runtime‚4095ý!Õ!ÝëM F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3668354666496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚46754778 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3852569967616 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚76304498 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3668158474240 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚305796894569472 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚76304498 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚70654497 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚292270660444160 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3852818371584 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3668354666496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚46754781 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚295918571187710 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23899716 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 % _timestamp‚1.7883933394351985e+09 _step‚361 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3647893704190 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 171241731 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚94937231 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚305796621261824 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚301944076197888 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚301944051294208 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚292270677483520 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚303205736637952 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3668158474240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚299537381971456 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚94937231 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 _runtime‚ 4095.19173064 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚70653967 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23899189 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3647701707442 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 171241731 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 171219105 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3668354666496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚46754778 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3852569967616 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚76304498 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3668158474240 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚305796894569472 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚76304498 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚70654497 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚292270660444160 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3852818371584 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3668354666496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚46754781 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚295918571187710 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚23899716 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 % _timestamp‚1.7883933394351985e+09 _step‚361 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3647893704190 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 171241731 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚94937231 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚305796621261824 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚301944076197888 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚301944051294208 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚292270677483520 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚303205736637952 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3668158474240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚299537381971456 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚94937231 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 _runtime‚ 4095.19173064 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚70653967 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚23899189 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3647701707442 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 171241731 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750éü9°J2ß- _wandbruntime‚4095  _runtime‚40953K!\àÖ 1loss/SRqqt1/detection_loss‚1.7056106020613473 .loss/FBt4/detection_loss‚2.435447896781721 0loss/FRt4/assignment_loss‚3.9326905552546183 0loss/SRqqt3/detection_loss‚2.244446077208588 % _timestamp‚1.7883933394370284e+09 2loss/SRqqt4/assignment_loss‚2.0394264973480913 0loss/FBt3/assignment_loss‚0.1989105713810854 /loss/FRt2/detection_loss‚1.6354740716230411 /loss/FBt1/detection_loss‚2.2249286876005283 epoch‚7 0loss/FRt1/assignment_loss‚3.0465029686400036 1loss/FBt1/assignment_loss‚0.10546180554622625 /loss/FRt4/detection_loss‚2.6046292662620543 1loss/SRqqt4/detection_loss‚2.3357210540596176 %loss/total_loss‚42.95528785430132 2loss/SRqqt1/assignment_loss‚1.5604811131142744 1loss/SRqqt2/assignment_loss‚0.901028148273463 /loss/FRt3/detection_loss‚2.5962141287791263 .loss/FBt3/detection_loss‚2.948891739760126 0loss/FRt2/assignment_loss‚1.5733514491182108 /loss/FRt3/assignment_loss‚3.502082926112336 1loss/FBt2/assignment_loss‚0.11409584802641767 .loss/FRt1/detection_loss‚2.286862358016875 /loss/FBt2/detection_loss‚0.4947868194045312 trainer/global_step‚4449 2loss/SRqqt3/assignment_loss‚1.4587388411164284 1loss/FBt4/assignment_loss‚0.14435164752955498 1loss/SRqqt2/detection_loss‚0.8651527812830487 _runtime‚4095.192837947 _step‚362p €1aáÛ % _timestamp‚1.7883933394370284e+09 2loss/SRqqt4/assignment_loss‚2.0394264973480913 0loss/FBt3/assignment_loss‚0.1989105713810854 /loss/FRt2/detection_loss‚1.6354740716230411 /loss/FBt1/detection_loss‚2.2249286876005283 epoch‚7 0loss/FRt1/assignment_loss‚3.0465029686400036 1loss/FBt1/assignment_loss‚0.10546180554622625 /loss/FRt4/detection_loss‚2.6046292662620543 1loss/SRqqt4/detection_loss‚2.3357210540596176 %loss/total_loss‚42.95528785430132 2loss/SRqqt1/assignment_loss‚1.5604811131142744 1loss/SRqqt2/assignment_loss‚0.901028148273463 /loss/FRt3/detection_loss‚2.5962141287791263 .loss/FBt3/detection_loss‚2.948891739760126 0loss/FRt2/assignment_loss‚1.5733514491182108 /loss/FRt3/assignment_loss‚3.502082926112336 1loss/FBt2/assignment_loss‚0.11409584802641767 .loss/FRt1/detection_loss‚2.286862358016875 /loss/FBt2/detection_loss‚0.4947868194045312 trainer/global_step‚4449 2loss/SRqqt3/assignment_loss‚1.4587388411164284 1loss/FBt4/assignment_loss‚0.14435164752955498 1loss/SRqqt2/detection_loss‚0.8651527812830487 _runtime‚4095.192837947 _step‚362 1loss/SRqqt1/detection_loss‚1.7056106020613473 .loss/FBt4/detection_loss‚2.435447896781721 0loss/FRt4/assignment_loss‚3.9326905552546183 0loss/SRqqt3/detection_loss‚2.244446077208588êp©Zþâ‚(:’ ŠïâÔ¥‚Õu gpu.0.gpu‚20.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.136' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.027199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0tâøRã‚(:Ç ŠïâÔÝ÷¼x network.sent‚ 9.233933e+06$ disk.nvme0n1.out‚45014.525390625" disk.nvme0c0n1.in‚ 261.28515625 disk.sda2.in‚0 cpu‚25.596038792798726 disk.rbd1.in‚0 disk.dm-0.in‚ 261.28515625 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 971.671875 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 proc.cpu.threads‚17) disk./.usagePercent‚71.53189127839266 disk.rbd1.out‚ 2.208984375 disk.nvme0n1.in‚ 261.28515625 disk.sda.out‚ 971.671875 disk.sda1.in‚0 disk.sda1.out‚0& disk.nvme0c0n1.out‚45014.525390625 disk.sda3.out‚ 971.671875 disk.rbd3.out‚ 2.349609375% disk./.usageGB‚1065.8768730163574 disk.rbd2.out‚364.4384765625 disk.sda.in‚ 0.1171875 disk.drbd1777.in‚0$ memory_percent‚92.12727546691895( proc.memory.availableMB‚ 1289.8671875# proc.memory.rssMB‚ 2894.45703125 disk.rbd3.in‚0! disk.dm-0.out‚45014.525390625 disk.sda3.in‚ 0.1171875 network.recv‚ 1.442287e+06 disk.rbd2.in‚0) proc.memory.percent‚17.66636371612549 disk.sda2.out‚0èo*ãä‚(:’ ™ïâÔˆ¬°u gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.479' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.495800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0墮=å‚(:² ™ïâÔÅÖ¼x disk.drbd1777.out‚0 disk.sda2.out‚0 disk.dm-0.out‚ 45113.9765625 proc.cpu.threads‚17 disk.rbd0.out‚ 0.2978515625$ disk.nvme0c0n1.out‚ 45113.9765625' proc.memory.availableMB‚ 1289.859375) proc.memory.percent‚17.66636371612549" disk.nvme0n1.out‚ 45113.9765625 disk.rbd0.in‚0 disk.sda.out‚985.25 disk.dm-1.out‚985.25# proc.memory.rssMB‚ 2894.45703125 network.sent‚ 9.310158e+06 network.recv‚ 1.447255e+06) disk./.usagePercent‚71.53167776810912 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 261.28515625$ disk./.usageGB‚1065.873691558838 disk.rbd2.out‚366.1962890625 disk.nvme0n1.in‚ 261.28515625" disk.nvme0c0n1.in‚ 261.28515625 disk.rbd3.in‚0 disk.sda3.in‚ 0.1171875$ memory_percent‚92.12732315063477 disk.rbd2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd1.out‚ 2.208984375 disk.sda.in‚ 0.1171875 disk.sda2.in‚0 cpu‚25.59611216958296 disk.sda3.out‚985.25áàM2æ- _wandbruntime‚4137  _runtime‚4137Ä–=[Éçà _step‚363 ' lr-AdamW/pg1‚0.00031883223065995236 ' lr-AdamW/pg2‚0.00031883223065995236 trainer/global_step‚4499 % _timestamp‚1.7883933814300516e+09 _runtime‚4137.186087629Š+Å­ÎèÈ _step‚363 ' lr-AdamW/pg1‚0.00031883223065995236 ' lr-AdamW/pg2‚0.00031883223065995236 trainer/global_step‚4499 % _timestamp‚1.7883933814300516e+09 _runtime‚4137.186087629ëLÀ¤2é- _wandbruntime‚4137  _runtime‚4137ÂioHè'êâO P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3707503629824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚77113980 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚95963285 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚95963285 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24155110 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3686629143902 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚306431798410752 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3707307450880 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚47246447 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚47246444 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚24154571 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚309048914473984 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚77113978 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚77113980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚306431585192448 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚71401557 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273320960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚302724294780928 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚305154825517568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚364 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3707503629824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚302724277741568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚77113978 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 173077265 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _runtime‚4137.187396076 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚306431585192448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚299073659432286 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚95940687 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3707307450880 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 173077265 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _runtime‚4137.187396076 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚306431585192448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚299073659432286 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚95940687 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3707307450880 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚95963285 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24155110 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3686629143902 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚306431798410752 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3707307450880 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚47246447 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚47246444 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚24154571 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚309048914473984 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚77113978 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚77113980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚306431585192448 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚71401557 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 273320960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚302724294780928 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚305154825517568ì˜Õk{2ì- _wandbruntime‚4137  _runtime‚4137 §®zò&íìM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚302789333817856 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚302789316778496 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚95984183 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚302789316778496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚24159865 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3687615023598 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚95961559 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3708105841664 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚77130500 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚77130498 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3894683995136 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚77130498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 % _timestamp‚1.7883933822088244e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3894926107648 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚71416343 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚309115277121536 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 173092057 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚306497635851776 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24160378 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚305220326110208 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚77130500 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚306497422620160 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚95984183 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚309115010105344 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 173114683 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚47256478 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3708105841664 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚365 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚302789316778496 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚24159865 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3687615023598 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚95961559 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3708105841664 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚77130500 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚77130498 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3894683995136 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚77130498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 % _timestamp‚1.7883933822088244e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3894926107648 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚71416343 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚309115277121536 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 173092057 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚306497635851776 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24160378 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚305220326110208 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚77130500 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚306497422620160 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚95984183 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚309115010105344 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 173114683 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚47256478 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3708105841664 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 _step‚365 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 EDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚47748144 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚526 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚309723697624576 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3747450997248 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3747254818304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚302293559016394 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚298566999517184 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 _runtime‚4179.922972782 _step‚368 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚312367006861824 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 250514432 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚312367282279936 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚96987639 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚305976229587968 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 174950217 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚97010237 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚309723484406272“Ióí'ýçO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚309723697624576 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3747450997248 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3747254818304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚302293559016394 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚298566999517184 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 _runtime‚4179.922972782 _step‚368 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚312367006861824 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 250514432 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚312367282279936 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚96987639 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚305976229587968 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 174950217 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚97010237 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚309723484406272 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3747254818304 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚47748144 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚526 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3936759335424 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚24418554 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 174987635 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 174987635 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚312433360261632 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚542 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3748249401344 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚77956498 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚72176732 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3748053209088 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _step‚369 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚306041285664256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚306041268624896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚308496625829888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚306041285664256 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚309789535065600 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3727336343006 trainer/global_step‚4549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚77956498 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚97031135 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚4180.703944319 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 % _timestamp‚1.7883934249474509e+09 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚77956500[÷ x¤€ñM MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚308496625829888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚306041285664256 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚309789535065600 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3727336343006 trainer/global_step‚4549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚77956498 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚97031135 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 _runtime‚4180.703944319 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 % _timestamp‚1.7883934249474509e+09 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚77956500 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚97008511 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚47758178 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3936759335424 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚24418554 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 174987635 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 174987635 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚312433360261632 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚542 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3748249401344 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚77956498 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚72176732 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3748053209088 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _step‚369 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚306041285664256 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚306041268624896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065ñ$iÌ2- _wandbruntime‚4180  _runtime‚4180žÒ2/Y‚Ó 0loss/FBt1/assignment_loss‚0.1370884492240272 0loss/FBt2/assignment_loss‚0.1446927240973518 0loss/SRqqt2/detection_loss‚0.766158261383972 _runtime‚4180.704986403 /loss/FRt3/assignment_loss‚3.368298263712363 0loss/FRt4/assignment_loss‚3.8016674834139206 /loss/FRt2/assignment_loss‚1.483704518790197 0loss/SRqqt3/detection_loss‚1.675760380923748 /loss/FBt3/detection_loss‚2.8184812728847777 .loss/FBt4/detection_loss‚2.677237529021043 0loss/FRt1/assignment_loss‚3.0564517521858217 2loss/SRqqt4/assignment_loss‚1.8966045020257725 /loss/FRt2/detection_loss‚1.7026159972930797 % _timestamp‚1.7883934249491613e+09 .loss/FRt1/detection_loss‚2.591386877695719 /loss/FBt2/detection_loss‚0.5158633350910821 2loss/SRqqt3/assignment_loss‚1.7215137048712317 .loss/FRt4/detection_loss‚2.510586583000772 1loss/SRqqt4/detection_loss‚2.3930145580978954 2loss/SRqqt2/assignment_loss‚0.9239261882807923 0loss/FBt3/assignment_loss‚0.5417056795808354 1loss/FBt4/assignment_loss‚0.07983832082335049 /loss/FBt1/detection_loss‚2.6057517752051353 %loss/total_loss‚43.27975787331954 _step‚370 /loss/FRt3/detection_loss‚2.5165055109695955 1loss/SRqqt1/detection_loss‚1.6423946284119366 epoch‚7 trainer/global_step‚4549 2loss/SRqqt1/assignment_loss‚1.7085095763351152â|ÿ^ƒØ .loss/FRt1/detection_loss‚2.591386877695719 /loss/FBt2/detection_loss‚0.5158633350910821 2loss/SRqqt3/assignment_loss‚1.7215137048712317 .loss/FRt4/detection_loss‚2.510586583000772 1loss/SRqqt4/detection_loss‚2.3930145580978954 2loss/SRqqt2/assignment_loss‚0.9239261882807923 0loss/FBt3/assignment_loss‚0.5417056795808354 1loss/FBt4/assignment_loss‚0.07983832082335049 /loss/FBt1/detection_loss‚2.6057517752051353 %loss/total_loss‚43.27975787331954 _step‚370 /loss/FRt3/detection_loss‚2.5165055109695955 1loss/SRqqt1/detection_loss‚1.6423946284119366 epoch‚7 trainer/global_step‚4549 2loss/SRqqt1/assignment_loss‚1.7085095763351152 0loss/FBt1/assignment_loss‚0.1370884492240272 0loss/FBt2/assignment_loss‚0.1446927240973518 0loss/SRqqt2/detection_loss‚0.766158261383972 _runtime‚4180.704986403 /loss/FRt3/assignment_loss‚3.368298263712363 0loss/FRt4/assignment_loss‚3.8016674834139206 /loss/FRt2/assignment_loss‚1.483704518790197 0loss/SRqqt3/detection_loss‚1.675760380923748 /loss/FBt3/detection_loss‚2.8184812728847777 .loss/FBt4/detection_loss‚2.677237529021043 0loss/FRt1/assignment_loss‚3.0564517521858217 2loss/SRqqt4/assignment_loss‚1.8966045020257725 /loss/FRt2/detection_loss‚1.7026159972930797 % _timestamp‚1.7883934249491613e+09ò¸×VQ„‚(:’ ÕïâÔÝù¶u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.895' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.979000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜ÚZ…‚(:Ï ÕïâԷ͆x disk.drbd1777.in‚0) disk./.usagePercent‚71.44280603263526 disk.rbd1.out‚ 2.208984375$ memory_percent‚92.12789535522461 proc.cpu.threads‚17 disk.sda1.out‚0 disk.drbd1777.out‚0 network.sent‚ 9.414956e+06 network.recv‚ 1.465666e+06% disk./.usageGB‚1064.5494384765625 disk.rbd3.out‚ 2.349609375& disk.nvme0n1.out‚45320.22607421875 disk.sda.out‚ 1035.5859375 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚ 0.1171875 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 261.2890625( disk.nvme0c0n1.out‚45320.22607421875' proc.memory.availableMB‚ 1289.765625) proc.memory.percent‚17.66636371612549 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚369.6767578125 disk.sda2.out‚0 disk.dm-0.in‚ 261.2890625 disk.rbd0.out‚ 0.2978515625 cpu‚25.596812805852263# disk.dm-0.out‚45320.22607421875 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 1035.5859375 disk.sda.in‚ 0.1171875 disk.sda2.in‚0 disk.dm-1.out‚ 1035.5859375 disk.nvme0n1.in‚ 261.2890625# proc.memory.rssMB‚ 2894.45703125øuò»†‚(:’ äïâÔÙ‚×u gpu.0.gpu‚19.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚150.912' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.182400000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0}w¬\7‡‚(:¬ äïâÔ¿õ²x disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.in‚ 261.2890625 proc.cpu.threads‚17) disk./.usagePercent‚71.44280603263526% disk./.usageGB‚1064.5494384765625 disk.rbd1.in‚0 disk.nvme0n1.in‚ 261.2890625 disk.drbd1777.out‚0$ disk.nvme0c0n1.out‚ 45418.5078125 disk.dm-0.out‚ 45418.5078125$ memory_percent‚92.12794303894043 disk.sda1.out‚0 disk.rbd1.out‚ 2.208984375" disk.nvme0n1.out‚ 45418.5078125 disk.sda.out‚1047 disk.sda1.in‚0 disk.drbd1777.in‚0 cpu‚25.597051970208057 disk.rbd3.in‚0 disk.sda3.in‚ 0.1171875 disk.sda3.out‚1047 disk.sda.in‚ 0.1171875 disk.rbd3.out‚ 2.349609375( proc.memory.availableMB‚ 1289.7578125 disk.rbd2.out‚371.1884765625 disk.rbd0.in‚0) proc.memory.percent‚17.66636371612549 disk.rbd2.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 261.2890625 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚1047# proc.memory.rssMB‚ 2894.45703125 network.sent‚ 9.502386e+06 network.recv‚ 1.471711e+06c^ L‘ˆ‚(:† óïâÔúôüu gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚91.583' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.3166 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ªawX‰‚(:Í óïâÔþÚÂx$ disk.nvme0n1.out‚45419.826171875 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-1.out‚ 1048.24609375 disk.rbd1.in‚0 disk.rbd2.out‚371.2900390625& disk.nvme0c0n1.out‚45419.826171875 cpu‚25.59711844276555 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.sda3.out‚ 1048.24609375 disk.rbd1.out‚ 2.208984375! disk.nvme0c0n1.in‚ 261.2890625# proc.memory.rssMB‚ 2894.45703125 proc.cpu.threads‚17) disk./.usagePercent‚71.44281012875582 disk.sda.in‚ 0.1171875) proc.memory.availableMB‚ 1289.74609375) proc.memory.percent‚17.66636371612549 network.recv‚ 1.476203e+06 disk.nvme0n1.in‚ 261.2890625 disk.sda.out‚ 1048.24609375% disk./.usageGB‚1064.5494995117188 disk.dm-0.in‚ 261.2890625 disk.rbd2.in‚0 disk.sda1.in‚0$ memory_percent‚92.12801456451416 network.sent‚ 9.508906e+06 disk.rbd3.out‚ 2.349609375! disk.dm-0.out‚45419.826171875 disk.sda3.in‚ 0.1171875 disk.dm-1.in‚ 0.1171875 disk.sda2.out‚0 disk.rbd0.out‚ 0.2978515625v—•Ð2Š- _wandbruntime‚4222  _runtime‚4222q¨3‹Ç‹Á _runtime‚4222.585417737 _step‚371 ' lr-AdamW/pg1‚0.00028114794861405345 ' lr-AdamW/pg2‚0.00028114794861405345 trainer/global_step‚4599 # _timestamp‚1.78839346682944e+09¾$ÌŒÆ # _timestamp‚1.78839346682944e+09 _runtime‚4222.585417737 _step‚371 ' lr-AdamW/pg1‚0.00028114794861405345 ' lr-AdamW/pg2‚0.00028114794861405345 trainer/global_step‚4599óVÿÊ2- _wandbruntime‚4222  _runtime‚4222_ } è'ŽâO JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚305513249564602 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚78765978 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚98057189 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24673861 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚309228198473728 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚301746968745984 _runtime‚4222.590025912 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚309228181434368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚315685298303488 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3766263779258 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚72923173 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3787202185728 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚532 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚309228198473728 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚48249847 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚311707375149568 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _step‚372 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚309228181434368 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3787398364672 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚98034591 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚315685580013056 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚78765978 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚311707350245888 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚305513040528702 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚98057189 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚98034591 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3978204863488 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚313015383620096 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚313015596838400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3787398364672 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚313015596838400 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚72923708 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3977948057600 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚78765980 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 256805888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3787202185728 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3766071782718 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚313015383620096 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 176823169 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚24673329 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚78765980 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 281709568 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 trainer/global_step‚4599¢h¢ÁçO eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚311707375149568 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _step‚372 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚309228181434368 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3787398364672 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚98034591 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚315685580013056 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚78765978 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚311707350245888 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚305513040528702 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚98057189 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚98034591 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3978204863488 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚313015383620096 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚313015596838400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3787398364672 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚313015596838400 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚72923708 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3977948057600 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚78765980 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 256805888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3787202185728 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3766071782718 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚313015383620096 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 176823169 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚535 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚24673329 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚78765980 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 281709568 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 trainer/global_step‚4599 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDKDä,DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚305513249564602 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚78765978 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚98057189 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24673861 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚309228198473728 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚301746968745984 _runtime‚4222.590025912 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚309228181434368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚315685298303488 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3766263779258 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚72923173 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3787202185728 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚532 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚309228198473728 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚48249847ô¢ul>2- _wandbruntime‚4223  _runtime‚4223˜5ºò&‘ìM NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚98055463 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚78782500 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚315751936053760 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚311772875742208 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3978787003904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3767057662414 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883934675966916e+09 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚301810568130560 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3766865665666 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24678979 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚309293237510656 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 176860587 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚48259878 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 176860587 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚98078087 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚72938860 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚72938345 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚4223.353358532 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚313081221047808 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚315751662746112 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚48259881 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 _step‚373 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚98078087 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚313081434279424 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 176837961 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚98055463 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚309293220471296 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚78782498 ß‘x÷&’ñM ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚309293220471296 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3788000576512 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3788196768768 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 248403968 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 _runtime‚4223.353358532 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚313081221047808 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚315751662746112 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚48259881 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 _step‚373 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚98078087 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚313081434279424 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 176837961 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚98055463 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚309293220471296 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚78782498 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚98055463 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚78782500 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚315751936053760 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚311772875742208 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3978787003904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3767057662414 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883934675966916e+09 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚301810568130560 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3766865665666 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24678979 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚309293237510656 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 176860587 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚48259878 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 273307648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 176860587 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚98078087 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚72938860 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚72938345 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚99104141 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3827149553152 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚314983625061888 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚304926955014144 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚316307282833920 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚308732731076910 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚312480133280768 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚48751544 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4020060116480 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚316307496052224 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚312480150320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24932309 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚99081543 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚514 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚308732940112810 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3827149553152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3805985098666 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚319003710082048 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 trainer/global_step‚4649 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚312480150320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚48751547 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 178696121 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3827345732096 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚79591978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 _step‚376 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚79591978 _runtime‚4265.612443221 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚314983649965568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 178673521 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 105092935688ßUí'¡çO PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚79591978 _runtime‚4265.612443221 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚314983649965568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 178673521 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚99104141 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚316307282833920 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚79591980 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚99104141 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3827149553152 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚314983625061888 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚304926955014144 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚316307282833920 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚308732731076910 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚312480133280768 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚48751544 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4020060116480 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚316307496052224 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚312480150320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚24932309 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚99081543 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚514 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚308732940112810 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3827149553152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3805985098666 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚319003710082048 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 trainer/global_step‚4649 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚312480150320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚48751547 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 178696121 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3827345732096 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚79591978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 _step‚376ø,ªi¤2¢- _wandbruntime‚4266  _runtime‚4266vYå¹ò&£ìM PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚316373333493248 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚315049150558208 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 178733539 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚79608500 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4020895805440 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 178733539 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 _runtime‚4266.391085549 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚316373120261632 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3828144136192 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3827947943936 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚99102415 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24937439 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚319070071267328 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3827947943936 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚48761578 trainer/global_step‚4649 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚312545172317696 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3828144136192 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚79608498 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚99125039 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚79608498 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚312545189357056 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚73698482 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚319069800056832 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚312545189357056 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚315049175461888 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚308797333380542 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚73699020 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3806778981822 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3806586985074 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 178710913 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚308797124344434 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚99102415 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680·&+h ¤ñM SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚312545172317696 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3828144136192 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚79608498 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚99125039 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚79608498 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚312545189357056 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚73698482 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚319069800056832 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚312545189357056 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚315049175461888 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚308797333380542 D>DeviceStatsMonitor.on_train_batch_end †ít/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚73699020 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3806778981822 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3806586985074 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 178710913 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚308797124344434 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚99102415 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚316373333493248 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚315049150558208 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 178733539 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚79608500 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4020895805440 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 178733539 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚538 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 _runtime‚4266.391085549 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚316373120261632 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3828144136192 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 246306816 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3827947943936 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚99102415 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚24937439 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚319070071267328 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3827947943936 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚48761578 trainer/global_step‚4649ù×k¾2¥- _wandbruntime‚4266  _runtime‚4266§‡[¦Õ 1loss/FBt2/assignment_loss‚0.14448605149114677 /loss/FRt4/detection_loss‚2.7675874752383076 .loss/FBt1/detection_loss‚2.363078382280138 .loss/FBt3/detection_loss‚2.176262676715851 epoch‚7 _runtime‚4266.392383325 2loss/SRqqt1/assignment_loss‚1.3654224822454364 2loss/SRqqt4/assignment_loss‚1.9981685011432722 /loss/FRt2/detection_loss‚1.7043872663480604 /loss/FBt2/detection_loss‚0.6212972202447846 /loss/FBt4/detection_loss‚2.9776910185813903 /loss/FRt1/detection_loss‚2.3140962133804956 0loss/FRt2/assignment_loss‚1.3553541231359172 2loss/SRqqt2/assignment_loss‚0.9084592783925145 0loss/SRqqt3/assignment_loss‚1.55881667373204 1loss/FBt4/assignment_loss‚0.01727570225193631 /loss/FRt3/detection_loss‚2.3330243285349073 0loss/SRqqt3/detection_loss‚1.853258817650304 % _timestamp‚1.7883935106365497e+09 0loss/FRt1/assignment_loss‚2.9921313597096337 /loss/FRt4/assignment_loss‚3.748227890460722 1loss/FBt3/assignment_loss‚0.14469377255752786 1loss/SRqqt1/detection_loss‚1.7624351742250897 %loss/total_loss‚41.97159860817281 1loss/SRqqt4/detection_loss‚2.3510986419824453 trainer/global_step‚4649 0loss/FBt1/assignment_loss‚0.0918043473664309 1loss/SRqqt2/detection_loss‚0.9278294366777391 /loss/FRt3/assignment_loss‚3.494711773826721 _step‚378ö2¯÷`§Ú 0loss/FRt2/assignment_loss‚1.3553541231359172 2loss/SRqqt2/assignment_loss‚0.9084592783925145 0loss/SRqqt3/assignment_loss‚1.55881667373204 1loss/FBt4/assignment_loss‚0.01727570225193631 /loss/FRt3/detection_loss‚2.3330243285349073 0loss/SRqqt3/detection_loss‚1.853258817650304 % _timestamp‚1.7883935106365497e+09 0loss/FRt1/assignment_loss‚2.9921313597096337 /loss/FRt4/assignment_loss‚3.748227890460722 1loss/FBt3/assignment_loss‚0.14469377255752786 1loss/SRqqt1/detection_loss‚1.7624351742250897 %loss/total_loss‚41.97159860817281 1loss/SRqqt4/detection_loss‚2.3510986419824453 trainer/global_step‚4649 0loss/FBt1/assignment_loss‚0.0918043473664309 1loss/SRqqt2/detection_loss‚0.9278294366777391 /loss/FRt3/assignment_loss‚3.494711773826721 _step‚378 1loss/FBt2/assignment_loss‚0.14448605149114677 /loss/FRt4/detection_loss‚2.7675874752383076 .loss/FBt1/detection_loss‚2.363078382280138 .loss/FBt3/detection_loss‚2.176262676715851 epoch‚7 _runtime‚4266.392383325 2loss/SRqqt1/assignment_loss‚1.3654224822454364 2loss/SRqqt4/assignment_loss‚1.9981685011432722 /loss/FRt2/detection_loss‚1.7043872663480604 /loss/FBt2/detection_loss‚0.6212972202447846 /loss/FBt4/detection_loss‚2.9776910185813903 /loss/FRt1/detection_loss‚2.3140962133804956ú¥øn¨‚(:’ ¯ðâÔ»øëu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚138.831' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.766199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚00&'U©‚(:Ê ¯ðâÔöÁþx disk.rbd1.in‚0 disk.sda.in‚ 0.1171875 disk.sda.out‚ 1103.14453125 disk.dm-0.in‚ 261.2890625# proc.memory.rssMB‚ 2894.45703125 disk.rbd3.in‚0 disk.dm-1.out‚ 1103.14453125$ memory_percent‚92.11728572845459 proc.cpu.threads‚17 disk.drbd1777.in‚0 network.recv‚ 1.494805e+06 disk.nvme0n1.in‚ 261.2890625 disk.dm-1.in‚ 0.1171875$ disk./.usageGB‚1064.549648284912# disk.nvme0n1.out‚45715.76953125 disk.sda3.in‚ 0.1171875 disk.rbd2.in‚0! disk.rbd2.out‚374.65380859375) proc.memory.percent‚17.66636371612549 disk.sda3.out‚ 1103.14453125 disk.drbd1777.out‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375! disk.nvme0c0n1.in‚ 261.2890625 disk.dm-0.out‚45715.76953125 network.sent‚ 9.613652e+06) disk./.usagePercent‚71.44282011304965% disk.nvme0c0n1.out‚45715.76953125 disk.sda2.out‚0 disk.rbd1.out‚ 2.208984375 cpu‚25.59789448386266 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625) proc.memory.availableMB‚ 1291.50390625Úôá^’ª‚(:‡ ¾ðâÔÜ‚²u gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.784' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.7568 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0š›T«‚(:É ¾ðâÔݾw disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.dm-0.in‚ 261.2890625 disk.dm-0.out‚ 45716.0703125 proc.cpu.threads‚17" disk.nvme0n1.out‚ 45716.0703125 disk.sda1.out‚0$ memory_percent‚92.11738109588623% disk./.usageGB‚1064.5496788024902 disk.dm-1.in‚ 0.1171875! disk.nvme0c0n1.in‚ 261.2890625) proc.memory.availableMB‚ 1291.48828125 network.recv‚ 1.500541e+06) disk./.usagePercent‚71.44282216110992 disk.rbd0.in‚0# proc.memory.rssMB‚ 2894.45703125 disk.rbd3.in‚0 disk.rbd1.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚376.22802734375 disk.nvme0n1.in‚ 261.2890625 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.dm-1.out‚ 1103.57421875 disk.sda.out‚ 1103.57421875 disk.drbd1777.in‚0$ disk.nvme0c0n1.out‚ 45716.0703125) proc.memory.percent‚17.66636371612549 cpu‚25.598090559525602 network.sent‚ 9.700548e+06 disk.sda3.out‚ 1103.57421875 disk.sda.in‚ 0.1171875 disk.rbd3.out‚ 2.349609375 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 2.208984375³[¤Ä‘¬‚(:† ÍðâÔò­£v gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.759' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.1518 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0íÜ‚)\­‚(:Ñ ÍðâÔºÀÌw disk.sda1.in‚0 disk.rbd0.in‚0" disk.nvme0c0n1.in‚ 261.37890625 disk.dm-0.in‚ 261.37890625 disk.sda3.in‚ 0.1171875) disk./.usagePercent‚71.44364368928731 disk.sda2.in‚0 cpu‚25.598209153466083 disk.drbd1777.out‚0& disk.nvme0c0n1.out‚45831.033203125 disk.dm-1.in‚ 0.1171875% disk./.usageGB‚1064.5619201660156! disk.dm-0.out‚45831.033203125! disk.rbd2.out‚377.84521484375$ memory_percent‚92.11735725402832 disk.rbd0.out‚ 0.2978515625 disk.rbd3.out‚ 2.349609375( proc.memory.availableMB‚ 1291.4921875 proc.cpu.threads‚17 disk.nvme0n1.in‚ 261.37890625$ disk.nvme0n1.out‚45831.033203125 disk.sda.in‚ 0.1171875 disk.dm-1.out‚ 1118.24609375 disk.rbd1.in‚0) proc.memory.percent‚17.66636371612549 network.recv‚ 1.504278e+06 disk.sda.out‚ 1118.24609375 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd3.in‚0 disk.sda3.out‚ 1118.24609375 disk.sda1.out‚0 disk.rbd2.in‚0# proc.memory.rssMB‚ 2894.45703125 disk.rbd1.out‚ 2.208984375 network.sent‚ 9.705969e+06j»°2®- _wandbruntime‚4308  _runtime‚4308#8ծǯÁ & lr-AdamW/pg1‚0.0002453941179567031 & lr-AdamW/pg2‚0.0002453941179567031 trainer/global_step‚4699 % _timestamp‚1.7883935525145345e+09 _runtime‚4308.270529486 _step‚379yÃègÌ°Æ & lr-AdamW/pg2‚0.0002453941179567031 trainer/global_step‚4699 % _timestamp‚1.7883935525145345e+09 _runtime‚4308.270529486 _step‚379 & lr-AdamW/pg1‚0.0002453941179567031ûòË‚>2±- _wandbruntime‚4308  _runtime‚4308,Ø´ì'²æO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚80417980 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚80417978 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚318259899877888 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚308106924242944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 % _timestamp‚1.7883935525155933e+09 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚311952630661018 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚322322127725056 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚315732102166528 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3867293099520 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚74445078 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚49253247 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3845706418074 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3867096920576 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚308106907203584 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3845514421534 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 180546473 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 100128495 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚25191834 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4061969206272 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚80417980 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 100151093 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚319599395266048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3867293099520 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 trainer/global_step‚4699 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3867096920576 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚322321869084160 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚311952421625118 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚49253244 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚319599182047744 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚80417978 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚315732085127168 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚318259924781568 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 _runtime‚4308.274034431 _step‚380 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚315732102166528 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚319599395266048 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 100128495 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4062202943488 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚319599182047744 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚74445603 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚25192356 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 100151093 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚315732102166528 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚319599395266048 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 100128495 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4062202943488 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚319599182047744 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚74445603 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚25192356 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 100151093 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚311952630661018 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚322322127725056 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚315732102166528 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3867293099520 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚74445078 XKDeviceStatsMonitor.on_train_batch_start/iºÌàc… nactive_split.large_pool.allocated‚49253247 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3845706418074 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3867096920576 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚308106907203584 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3845514421534 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 180546473 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 100128495 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚25191834 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4061969206272 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚80417980 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 100151093 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚319599395266048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0ü¾YIó2´- _wandbruntime‚4309  _runtime‚4309·¤ž¦ö&µðM [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚315797141203456 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 100171991 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚308170523627520 _step‚381 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3867895311360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚49263278 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚312017023928750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚74460655 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3868091503616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚319665232707072 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚25197374 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚318325450277888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚80434500 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚308170506588160 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3846500301230 % _timestamp‚1.7883935532747364e+09 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚74460125 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 100149367 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚315797141203456 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚312016814892642 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚25196847 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 180606491 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3867895311360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚80434498 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚322388249189888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚4699 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3868091503616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 180606491 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚315797124164096 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚80434498 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚319665232707072 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚80434500 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚315797124164096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0pB_û&¶õM WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚312017023928750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚74460655 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3868091503616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚319665232707072 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚25197374 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚318325450277888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚80434500 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚308170506588160 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3846500301230 % _timestamp‚1.7883935532747364e+09 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚74460125 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 100149367 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚315797141203456 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚312016814892642 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚25196847 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 180606491 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3867895311360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚80434498 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚322388249189888 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚4699 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3868091503616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 180606491 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚315797124164096 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚80434498 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚319665232707072 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚80434500 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚315797124164096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚315797141203456 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 100171991 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚308170523627520 _step‚381 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3867895311360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚49263278 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 229529600 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0ýÊ.й2·- _wandbruntime‚4309  _runtime‚4309äþ°Y¸Ó epoch‚7 $ _timestamp‚1.788393553276484e+09 %loss/total_loss‚42.74559451430039 0loss/FRt2/assignment_loss‚1.4446334698290195 2loss/SRqqt1/assignment_loss‚1.6777505199566032 1loss/FBt3/assignment_loss‚0.28878687006155296 .loss/FRt1/detection_loss‚2.467797772786052 0loss/SRqqt3/detection_loss‚1.961602988525441 1loss/SRqqt4/detection_loss‚2.3064200815180658 _step‚382 /loss/FBt1/detection_loss‚2.2912718772888185 /loss/FBt4/detection_loss‚2.5437205465216386 trainer/global_step‚4699 .loss/FRt4/detection_loss‚2.502676078790351 _runtime‚4309.032261686 /loss/FBt2/detection_loss‚0.5210502953323696 /loss/FRt4/assignment_loss‚3.652628688872615 1loss/SRqqt3/assignment_loss‚1.612483569665959 2loss/SRqqt4/assignment_loss‚1.9135369123812294 0loss/FBt1/assignment_loss‚0.1483824398892466 .loss/FRt3/detection_loss‚2.510462631146575 0loss/SRqqt2/detection_loss‚1.052514359715485 2loss/SRqqt2/assignment_loss‚0.9063590074618978 /loss/FBt3/detection_loss‚2.4214207657745908 0loss/FBt2/assignment_loss‚0.1667034847253347 2loss/FBt4/assignment_loss‚0.050280624129012914 /loss/FRt2/detection_loss‚1.6234995637370169 0loss/FRt1/assignment_loss‚3.4996146074275383 /loss/FRt3/assignment_loss‚3.394538636124411 1loss/SRqqt1/detection_loss‚1.7874587226395655vù<^¹Ø /loss/FRt4/assignment_loss‚3.652628688872615 1loss/SRqqt3/assignment_loss‚1.612483569665959 2loss/SRqqt4/assignment_loss‚1.9135369123812294 0loss/FBt1/assignment_loss‚0.1483824398892466 .loss/FRt3/detection_loss‚2.510462631146575 0loss/SRqqt2/detection_loss‚1.052514359715485 2loss/SRqqt2/assignment_loss‚0.9063590074618978 /loss/FBt3/detection_loss‚2.4214207657745908 0loss/FBt2/assignment_loss‚0.1667034847253347 2loss/FBt4/assignment_loss‚0.050280624129012914 /loss/FRt2/detection_loss‚1.6234995637370169 0loss/FRt1/assignment_loss‚3.4996146074275383 /loss/FRt3/assignment_loss‚3.394538636124411 1loss/SRqqt1/detection_loss‚1.7874587226395655 epoch‚7 $ _timestamp‚1.788393553276484e+09 %loss/total_loss‚42.74559451430039 0loss/FRt2/assignment_loss‚1.4446334698290195 2loss/SRqqt1/assignment_loss‚1.6777505199566032 1loss/FBt3/assignment_loss‚0.28878687006155296 .loss/FRt1/detection_loss‚2.467797772786052 0loss/SRqqt3/detection_loss‚1.961602988525441 1loss/SRqqt4/detection_loss‚2.3064200815180658 _step‚382 /loss/FBt1/detection_loss‚2.2912718772888185 /loss/FBt4/detection_loss‚2.5437205465216386 trainer/global_step‚4699 .loss/FRt4/detection_loss‚2.502676078790351 _runtime‚4309.032261686 /loss/FBt2/detection_loss‚0.5210502953323696þî­Zœº‚(:‘ ÜðâԞϮv gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.08' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.216000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ì‚Â,[»‚(:Ð ÜðâÔ¶ÓÖx disk.drbd1777.in‚0 disk.sda3.in‚ 0.1171875 disk.nvme0n1.in‚ 261.37890625# proc.memory.rssMB‚ 2894.45703125) proc.memory.percent‚17.66636371612549 disk.sda.in‚ 0.1171875 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375 disk.dm-1.in‚ 0.1171875 disk.rbd1.in‚0 disk.rbd2.in‚0 proc.cpu.threads‚17& disk.nvme0c0n1.out‚45930.337890625 disk.dm-0.in‚ 261.37890625 cpu‚25.59842813404677 network.sent‚ 9.71209e+06% disk./.usageGB‚1064.5589065551758! disk.dm-0.out‚45930.337890625 disk.sda3.out‚ 1133.80859375$ disk.nvme0n1.out‚45930.337890625) proc.memory.availableMB‚ 1291.20703125 network.recv‚ 1.509059e+06 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 disk.sda2.out‚0) disk./.usagePercent‚71.44344144333526 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 261.37890625$ memory_percent‚92.11909770965576 disk.sda.out‚ 1133.80859375 disk.sda1.out‚0 disk.rbd3.in‚0 disk.rbd1.out‚ 2.208984375 disk.rbd0.in‚0 disk.dm-1.out‚ 1133.80859375! disk.rbd2.out‚377.84521484375MkX’¼‚(:‡ ëðâÔÕà­u gpu.0.gpu‚30.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚156.568' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.3136 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢˜•>X½‚(:Í ëðâÔ¥ç¨w% disk./.usageGB‚1064.5588874816895 disk.nvme0n1.in‚ 261.37890625 disk.sda1.in‚0% disk.nvme0n1.out‚45932.7373046875 disk.sda.out‚ 1149.359375 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 261.37890625 disk.sda3.out‚ 1149.359375 disk.sda1.out‚0) proc.memory.percent‚17.66636371612549 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚ 0.1171875 disk.dm-1.out‚ 1149.359375 disk.rbd1.in‚0 disk.sda.in‚ 0.1171875# proc.memory.rssMB‚ 2894.45703125$ memory_percent‚92.11912155151367' proc.memory.availableMB‚ 1291.203125 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375" disk.dm-0.out‚45932.7373046875 disk.sda3.in‚ 0.1171875 disk.rbd1.out‚ 2.208984375 proc.cpu.threads‚17 network.sent‚ 9.799918e+06 network.recv‚ 1.515195e+06) disk./.usagePercent‚71.44344016329758" disk.nvme0c0n1.in‚ 261.37890625 disk.sda2.in‚0! disk.rbd2.out‚379.45849609375 cpu‚25.598618708401816' disk.nvme0c0n1.out‚45932.7373046875 disk.rbd2.in‚0->…°’¾‚(:‡ úðâÔ•þÇu gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.264' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.2528 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)DpX¿‚(:Í úðâÔÀx% disk./.usageGB‚1064.5588874816895 disk.sda2.out‚0 disk.sda1.out‚0" disk.nvme0c0n1.in‚ 261.37890625! disk.dm-0.out‚46031.076171875 disk.sda3.in‚ 0.1171875 disk.sda3.out‚ 1149.80078125$ memory_percent‚92.11759567260742' proc.memory.availableMB‚ 1291.453125# proc.memory.rssMB‚ 2894.45703125 disk.dm-1.in‚ 0.1171875 disk.drbd1777.in‚0$ disk.nvme0n1.out‚46031.076171875 disk.sda.in‚ 0.1171875 disk.sda1.in‚0! disk.rbd2.out‚379.45849609375 cpu‚25.59877758292052 proc.cpu.threads‚17& disk.nvme0c0n1.out‚46031.076171875 disk.dm-0.in‚ 261.37890625) proc.memory.percent‚17.66636371612549 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.rbd1.out‚ 2.208984375 network.recv‚ 1.5188e+06) disk./.usagePercent‚71.44344016329758 disk.sda.out‚ 1149.80078125 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375 disk.dm-1.out‚ 1149.80078125 disk.rbd1.in‚0 network.sent‚ 9.805333e+06 disk.nvme0n1.in‚ 261.37890625 disk.rbd2.in‚0 disk.rbd3.in‚0 Û3½2À- _wandbruntime‚4351  _runtime‚4351-ªæÈÁ $ _timestamp‚1.788393595333715e+09 _runtime‚4351.090982344 _step‚383 ' lr-AdamW/pg1‚0.00022827869193450714 ' lr-AdamW/pg2‚0.00022827869193450714 trainer/global_step‚4749ÖèÍÂÇ ' lr-AdamW/pg1‚0.00022827869193450714 ' lr-AdamW/pg2‚0.00022827869193450714 trainer/global_step‚4749 $ _timestamp‚1.788393595333715e+09 _runtime‚4351.090982344 _step‚383ÿT¬ª÷2Ã- _wandbruntime‚4351  _runtime‚4351,â‰U- ÄæO _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4104107394560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚508 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚81243978 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚311286893471744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 101175447 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚315172112173326 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚325640540729344 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 VBDeðs0!¿viceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚322891294479872 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚318984054012928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3907044288000 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 101175447 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚81243980 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚318984054012928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3907240466944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚322891081261568 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _step‚384 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚325640282088448 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3885427737482 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚318984036973568 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚318984036973568 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3907044288000 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚322891081261568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚25449897 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 182442025 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚25450402 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚75205349 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚321536174693888 trainer/global_step‚4749 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚75204841 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚49754944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚321536199597568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚322891294479872 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4104341131776 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 182419425 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚505 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚311286876432384 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚81243978 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3885235740942 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚49754947oO@Ÿñ'ÅëO ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3885235740942 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚49754947 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4104107394560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚508 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚81243978 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚311286893471744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 101175447 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚315172112173326 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚325640540729344 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚322891294479872 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚318984054012928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3907044288000 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 101175447 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚81243980 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚318984054012928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3907240466944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚322891081261568 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _step‚384 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚325640282088448 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3885427737482 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚318984036973568 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚318984036973568 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3907044288000 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚322891081261568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚25449897 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 182442025 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚25450402 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚75205349 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚321536174693888 trainer/global_step‚4749 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚75204841 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚49754944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚321536199597568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚322891294479872 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4104341131776 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 182419425 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚505 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚311286876432384 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚81243978€È5(2Æ- _wandbruntime‚4351  _runtime‚4351®!8Áõ&ÇïM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚4749 _step‚385 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚81260498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚322956918689280 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚49764978 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚4351.860933178 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚319049076010496 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚315236714476958 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚49764981 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚75219927 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 $ _timestamp‚1.788393596103492e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 182479443 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 101218943 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 182479443 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 101218943 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 182456817 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3886221620638 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚321601700190208 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚515 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚75220445 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚322957131920896 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚311350492856320 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 101196319 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚311350475816960 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0Áð-&ÈôM UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 182479443 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 101218943 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 182456817 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3886221620638 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚321601700190208 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚515 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚75220445 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚322957131920896 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚311350492856320 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 101196319 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚311350475816960 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?Devic»J ôeStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚4749 _step‚385 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚81260498 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚322956918689280 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚49764978 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _runtime‚4351.860933178 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚319049076010496 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚315236714476958 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚518 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚49764981 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚75219927 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 $ _timestamp‚1.788393596103492e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 182479443 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 101218943 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4146364745216 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 102222399 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚82069980 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 102222399 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3946991655424 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚328958576323584 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚326182980475392 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3946991655424 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚318391802721534 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚525 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4146364745216 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 102222399 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚82069980 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 102222399 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3946991655424 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚328958576323584 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚326182980475392 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3946991655424 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚318391802721534 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚82086500 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚25712260 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚82086498 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 102243271 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚324877999909888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 184329769 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3947986238464 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3947986238464 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _step‚389 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚25712790 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚329024935818240 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4146960812032 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚314530445045760 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚50266678 _runtime‚4395.075813264 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚322301044896256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚322301027856896 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 102265895 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚75978938 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚4799 # _timestamp‚1.78839363931942e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3947790046208 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 184352395 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4147202924544 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚322301027856896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚322301044896256 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 184352395 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚533 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚314530462085120ZfÄù&ÚóM C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚533 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚314530462085120 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚82086500 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚25712260 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚82086498 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 102243271 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚324877999909888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 184329769 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3947986238464 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3947986238464 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _step‚389 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚25712790 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚329024935818240 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4146960812032 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚314530445045760 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚50266678 _runtime‚4395.075813264 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚322301044896256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚322301027856896 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 102265895 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚75978938 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚4799 # _timestamp‚1.78839363931942e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3947790046208 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 184352395 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4147202924544 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚322301027856896 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚322301044896256 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 184352395…¶pÌ2Û- _wandbruntime‚4395  _runtime‚4395i£a5WÜÑ 1loss/SRqqt2/detection_loss‚1.0130996906067715 /loss/FBt3/detection_loss‚2.5971715624133744 /loss/FRt3/assignment_loss‚3.150754307707151 1loss/SRqqt3/detection_loss‚1.8081062814181155 epoch‚7 0loss/FRt4/assignment_loss‚3.4289633003441065 2loss/SRqqt1/assignment_loss‚1.4219070202217692 0loss/SRqqt4/detection_loss‚2.351109733184179 /loss/FBt2/detection_loss‚0.6363963250325815 /loss/FBt4/detection_loss‚2.5204842319855323 %loss/total_loss‚41.69458972566743 /loss/SRqqt3/assignment_loss‚1.5415300347228 /loss/FRt4/detection_loss‚2.5819606104412594 0loss/SRqqt1/detection_loss‚1.639391142050777 .loss/FBt1/detection_loss‚2.167568251490593 /loss/FRt1/assignment_loss‚3.047045720012292 trainer/global_step‚4799 2loss/SRqqt4/assignment_loss‚1.7736697311675738 /loss/FRt1/detection_loss‚2.3055934434675653 -loss/FRt3/detection_loss‚2.32124103885144 1loss/FBt2/assignment_loss‚0.10563706708419214 0loss/FBt4/assignment_loss‚0.3934506520230431 /loss/FRt2/detection_loss‚1.6136741549293003 0loss/FRt2/assignment_loss‚1.6355109692800163 1loss/FBt1/assignment_loss‚0.24592126271454617 % _timestamp‚1.7883936393212626e+09 _runtime‚4395.077109316 _step‚390 2loss/SRqqt2/assignment_loss‚1.0518405460216171 0loss/FBt3/assignment_loss‚0.34256264849682348ÕÞ\ÝÖ 2loss/SRqqt2/assignment_loss‚1.0518405460216171 0loss/FBt3/assignment_loss‚0.3425626484968234 1loss/SRqqt2/detection_loss‚1.0130996906067715 /loss/FBt3/detection_loss‚2.5971715624133744 /loss/FRt3/assignment_loss‚3.150754307707151 1loss/SRqqt3/detection_loss‚1.8081062814181155 epoch‚7 0loss/FRt4/assignment_loss‚3.4289633003441065 2loss/SRqqt1/assignment_loss‚1.4219070202217692 0loss/SRqqt4/detection_loss‚2.351109733184179 /loss/FBt2/detection_loss‚0.6363963250325815 /loss/FBt4/detection_loss‚2.5204842319855323 %loss/total_loss‚41.69458972566743 /loss/SRqqt3/assignment_loss‚1.5415300347228 /loss/FRt4/detection_loss‚2.5819606104412594 0loss/SRqqt1/detection_loss‚1.639391142050777 .loss/FBt1/detection_loss‚2.167568251490593 /loss/FRt1/assignment_loss‚3.047045720012292 trainer/global_step‚4799 2loss/SRqqt4/assignment_loss‚1.7736697311675738 /loss/FRt1/detection_loss‚2.3055934434675653 -loss/FRt3/detection_loss‚2.32124103885144 1loss/FBt2/assignment_loss‚0.10563706708419214 0loss/FBt4/assignment_loss‚0.3934506520230431 /loss/FRt2/detection_loss‚1.6136741549293003 0loss/FRt2/assignment_loss‚1.6355109692800163 1loss/FBt1/assignment_loss‚0.24592126271454617 % _timestamp‚1.7883936393212626e+09 _runtime‚4395.077109316 _step‚390†ˆ‘Þ‚(:† ¶ñâÔçÌÊu gpu.0.gpu‚3.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚110.158' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.0316 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“/4`Vß‚(:Ë ¶ñâÔîœÃx( disk.nvme0c0n1.out‚47590.08154296875 disk.rbd3.in‚0 disk.dm-0.in‚ 265.54296875 disk.rbd2.in‚0% disk./.usageGB‚1065.8851127624512 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.out‚ 2.349609375! disk.rbd2.out‚384.32958984375# proc.memory.rssMB‚ 2894.45703125) disk./.usagePercent‚71.53244425466663 disk.sda.in‚0.125 disk.sda.out‚ 1186.22265625 disk.dm-1.out‚ 1186.22265625$ memory_percent‚92.12038516998291) proc.memory.percent‚17.66636371612549 disk.nvme0n1.in‚ 265.54296875 disk.drbd1777.out‚0 disk.sda3.in‚0.125 disk.rbd1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda3.out‚ 1186.22265625 proc.cpu.threads‚17 network.recv‚ 1.53763e+06 disk.rbd0.in‚0" disk.nvme0c0n1.in‚ 265.54296875 cpu‚25.599458106103185) proc.memory.availableMB‚ 1290.99609375& disk.nvme0n1.out‚47590.08154296875# disk.dm-0.out‚47590.08154296875 disk.dm-1.in‚0.125 disk.rbd1.out‚ 2.208984375 network.sent‚ 9.979041e+06 disk.rbd0.out‚ 0.2978515625]9dsà‚(:… ÅñâÔÀƒªu gpu.0.gpu‚3.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.772' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7544 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ç GºPá‚(:Å ÅñâÔŠÆx disk.dm-1.out‚ 1202.9453125 disk.sda.in‚0.125 disk.sda1.in‚0 disk.drbd1777.in‚0" disk.nvme0c0n1.in‚ 265.54296875 disk.rbd3.in‚0 disk.dm-0.in‚ 265.54296875# disk.dm-0.out‚47688.68896484375 network.recv‚ 1.542832e+06% disk./.usageGB‚1065.8845710754395( disk.nvme0c0n1.out‚47688.68896484375 proc.memory.rssMB‚ 2893.59375 network.sent‚ 1.0055399e+07 disk.sda.out‚ 1202.9453125 disk.sda1.out‚0' proc.memory.availableMB‚ 1291.640625* proc.memory.percent‚17.661094665527344) disk./.usagePercent‚71.53240790159676 disk.dm-1.in‚0.125 disk.sda2.in‚0$ memory_percent‚92.11645126342773 proc.cpu.threads‚17 disk.rbd2.in‚0! disk.rbd2.out‚384.43505859375 disk.sda3.out‚ 1202.9453125 cpu‚25.60315770202104 disk.rbd1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0 disk.rbd1.out‚ 2.294921875 disk.nvme0n1.in‚ 265.54296875 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375& disk.nvme0n1.out‚47688.68896484375 disk.rbd0.in‚0 disk.sda3.in‚0.125&™¦ šâ‚(: ÔñâÔ¥æ¯u gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.58' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚17.715999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÉŸ"zUã‚(:Ê ÔñâÔ¸†¼x disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0 cpu‚25.623888105590805 disk.dm-1.out‚ 1220.87890625 network.sent‚ 1.0061414e+07 network.recv‚ 1.547391e+06 disk.sda3.in‚0.125 disk.sda3.out‚ 1220.87890625 disk.sda.in‚0.125 disk.sda2.in‚0 disk.rbd3.in‚0 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 265.54296875$ disk./.usageGB‚1065.884780883789 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.54296875 disk.sda1.out‚0) proc.memory.availableMB‚ 1292.44921875 proc.cpu.threads‚17) disk./.usagePercent‚71.53242198201116 disk.dm-1.in‚0.125& disk.nvme0n1.out‚47786.98193359375 disk.rbd0.in‚0# disk.dm-0.out‚47786.98193359375 disk.sda.out‚ 1220.87890625 disk.drbd1777.out‚0$ memory_percent‚92.11151599884033! proc.memory.rssMB‚ 2892.734375 disk.rbd3.out‚ 2.349609375 disk.rbd1.in‚0 disk.rbd1.out‚ 2.357421875! disk.rbd2.out‚385.94677734375 disk.drbd1777.in‚0( disk.nvme0c0n1.out‚47786.98193359375) proc.memory.percent‚17.65584945678711 disk.dm-0.in‚ 265.54296875©g/C‘ä‚(:† ãñâÔÌÁ¶w gpu.0.gpu‚70.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚31.0 gpu.0.powerWatts‚88.979' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.7958 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ãÐópSå‚(:È ãñâÔöìÒy" disk.nvme0c0n1.in‚ 265.54296875 disk.rbd3.in‚0 disk.dm-0.in‚ 265.54296875 disk.sda3.out‚ 1236.5546875 cpu‚25.644905962359292 network.sent‚ 1.0068244e+07 disk.nvme0n1.in‚ 265.54296875& disk.nvme0n1.out‚47787.79638671875 disk.drbd1777.out‚0 disk.sda3.in‚0.125 disk.dm-1.out‚ 1236.5546875$ memory_percent‚92.57047176361084 disk.sda1.in‚0 disk.rbd0.in‚0 network.recv‚ 1.552421e+06 disk.sda.out‚ 1236.5546875 disk.drbd1777.in‚0# disk.dm-0.out‚47787.79638671875 disk.dm-1.in‚0.125) proc.memory.availableMB‚ 1217.25390625 disk.rbd1.out‚ 2.357421875 proc.cpu.threads‚17% disk./.usageGB‚1065.8831329345703 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.sda1.out‚0( disk.nvme0c0n1.out‚47787.79638671875 disk.rbd3.out‚ 2.349609375 proc.memory.rssMB‚ 2968.40625) disk./.usagePercent‚71.53231138675635! disk.rbd2.out‚386.11083984375* proc.memory.percent‚18.117713928222656 disk.sda.in‚0.125 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.sda2.out‚0VJ©2æ- _wandbruntime‚4464  _runtime‚4464õ§{#õ 給 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4519194960594177 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2830807864665985 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.10990338772535324 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.3317340612411499 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.44583335518836975 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.34258317947387695 jet/accuracy_7_of_7‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 8"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.464457243680954 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1312144696712494 (particle/f_score‚0.41921937465667725 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3399471044540405 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974355459213257 jet/accuracy_5_of_6‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.47200462222099304 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3684210479259491 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 8"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.348710298538208 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3754992187023163 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.19443166255950928 8 Purity/2FRt2SRqqt*FBt/FR1æ­çùt_purity‚0.13534174859523773 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3995591998100281 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.29642099142074585 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ,jet/accuracy_4_of_5‚0.002481389557942748 particle/accuracy_5_of_5‚1 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.38927754759788513 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36314472556114197 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09264875203371048 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4304564893245697 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.1560983806848526 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.48469242453575134 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.05592592805624008 ,jet/accuracy_4_of_4‚0.003168202703818679 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.02850494347512722 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3710178732872009 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12471263855695724 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.49316784739494324 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12019733339548111 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23172755539417267 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.4122837781906128 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9961372017860413 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.14853380620479584 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.14140497148036957 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8564716577529907 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 +"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.25 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.44809868931770325 particle/accuracy_3_of_5‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 8 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.02380952425301075 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0357142873108387 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4634985327720642 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ( Purity/3FRt1SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5175719261169434 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.4340464472770691 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.47813814878463745 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.19255951046943665 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.14305555820465088 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2777777910232544 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2348484843969345 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.6904762387275696 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2506897449493408 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4515853822231293 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09157183766365051 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1941763460636139 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 jet/accuracy_6_of_6‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.14334282279014587 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4601907432079315 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9928579926490784 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9222223162651062 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.40833333134651184 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1386958211660385 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9892777800559998 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48571476340293884 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9955452680587769 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4336555004119873 particle/accuracy_2_of_3‚1 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.390625 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6790979504585266 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4377272427082062 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.15045858919620514 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2998954951763153 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9900306463241577 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40804848074913025 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18987415730953217 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.47004571557044983 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02648942731320858 particle/accuracy_5_of_7‚1 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4610220491886139 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.31440478563308716 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.2076190561056137 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_1_of_6‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 % _timestamp‚1.7883937090697289e+09 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9892541766166687 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.993921160697937 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.48066726326942444 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.12404762953519821 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.45030680298805237 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.19901201128959656 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19477839767932892 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5777778029441833 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.023403571918606758 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.06328042596578598 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.40743696689605713 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9934173822402954 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.061511993408203125 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.44136562943458557 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3532727360725403 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_2_of_5‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.46666663885116577 _step‚391 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.16744762659072876 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.375 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 particle/accuracy_1_of_7‚1 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.29979708790779114 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3130760192871094 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11342593282461166 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 trainer/global_step‚4831 :"Purity/2FRt0SRqqt0FBt/event_purity‚0.01506772730499506 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3698415458202362 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9913756847381592 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8991770148277283 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4519194960594177 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3296296298503876 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.27629798650741577 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1553741991519928 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.21054469048976898 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4653668701648712 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 particle/accuracy_4_of_7‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.19585253298282623 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2694639563560486 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_7‚1 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22186264395713806 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.445833295583725 +jet/accuracy_4_of_6‚0.07894736528396606 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07970396429300308 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.992557168006897 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9166666865348816 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_1‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.18666668236255646 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.09283608198165894 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44688478112220764 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9898256659507751 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41729649901390076 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4095747470855713 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.026581183075904846 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3408532440662384 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 )jet/accuracy_1_of_1‚0.456368625164032 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.19266614317893982 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *particle/specificity‚0.953597366809845 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42856690287590027 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.44394436478614807 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 epoch‚7 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1612892895936966 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.25099843740463257 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 *jet/accuracy_2_of_6‚0.4298245310783386 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.10255540162324905 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_6_of_6‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 _runtime‚4464.827933676 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3668217360973358 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4466589689254761 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5030864477157593 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.4325714111328125 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.41050171852111816 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8267597556114197 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.20019203424453735 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11948460340499878 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3076923191547394 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3571428656578064 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8846477270126343 particle/accuracy_2_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.42185521125793457 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.42171716690063477 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.47101452946662903 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.20725905895233154 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3706369996070862 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9811885356903076 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.350166380405426 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 )jet/accuracy_2_of_4‚0.346370130777359 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.21399180591106415 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.22111640870571136 particle/accuracy_1_of_5‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 particle/accuracy_4_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13424888253211975 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005971279926598072 particle/accuracy_2_of_2‚1 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4542991816997528 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_2_of_4‚1 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_4‚0.05535747855901718 particle/accuracy_1_of_3‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19455312192440033 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.45026135444641113 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1568627655506134 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1619195193052292 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.47271162271499634 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2788989841938019 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.21807622909545898 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4384588301181793 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.006451612804085016 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.14814814925193787 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02688361145555973 particle/accuracy_3_of_3‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 jet/accuracy_4_of_7‚0 8 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.02380952425301075 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4173242449760437 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9255732893943787 particle/accuracy_1_of_2‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.23427610099315643 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.13995949923992157 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.2282467484474182 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.003225806402042508 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06913421303033829 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.42732468247413635 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5080357789993286 +jet/accuracy_3_of_3‚0.02430657669901848 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 +jet/accuracy_3_of_6‚0.17543859779834747 jet/accuracy_2_of_7‚0 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9978864192962646 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1504475623369217 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15591691434383392 particle/accuracy_1_of_4‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 *loss/val_total_loss‚42.669207364237906 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18431474268436432 particle/accuracy_6_of_7‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.42467206716537476 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.42105263471603394 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4950680434703827 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.6904762387275696 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9848148226737976 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1900569051504135 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986535906791687 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.1891123205423355 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.056269340217113495 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010485708713531494 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5151331424713135 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3909013569355011 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.010335271246731281 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.21720990538597107 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9938075542449951 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.927579402923584 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1570652723312378 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.031105993315577507 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13712941110134125 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.7179811000823975 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4466666579246521 jet/accuracy_6_of_7‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.14725494384765625 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3421022891998291 jet/accuracy_1_of_7‚1 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4121406376361847 :"P[ëH(Nurity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42551207542419434 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004966773558408022 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.4125405251979828 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.43707752227783203 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.14140497148036957 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4583333432674408 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8392475247383118 jet/accuracy_5_of_5‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2944444417953491 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 particle/accuracy_2_of_7‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30282285809516907 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_4‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.37024396657943726 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9894065856933594 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6785714030265808 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4769119918346405 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6371600031852722 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4585232436656952<\¤w躦 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5080357789993286 +jet/accuracy_3_of_3‚0.02430657669901848 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 +jet/accuracy_3_of_6‚0.17543859779834747 jet/accuracy_2_of_7‚0 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9978864192962646 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1504475623369217 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15591691434383392 particle/accuracy_1_of_4‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 *loss/val_total_loss‚42.669207364237906 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18431474268436432 particle/accuracy_6_of_7‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.42467206716537476 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.42105263471603394 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4950680434703827 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.6904762387275696 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9848148226737976 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1900569051504135 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986535906791687 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.1891123205423355 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.056269340217113495 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010485708713531494 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_3_of_7‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5151331424713135 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3909013569355011 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.010335271246731281 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.21720990538597107 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9938075542449951 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.927579402923584 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1570652723312378 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.031105993315577507 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13712941110134125 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.7179811000823975 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4466666579246521 jet/accuracy_6_of_7‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.14725494384765625 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3421022891998291 jet/accuracy_1_of_7‚1 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4121406376361847 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42551207542419434 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004966773558408022 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.4125405251979828 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.43707752227783203 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.14140497148036957 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4583333432674408 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8392475247383118 jet/accuracy_5_of_5‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2944444417953491 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 particle/accuracy_2_of_7‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30282285809516907 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_4‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.37024396657943726 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9894065856933594 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6785714030265808 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4769119918346405 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6371600031852722 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4585232436656952 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4519194960594177 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2830807864665985 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.10990338772535324 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.3317340612411499 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.44583335518836975 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.34258317947387695 jet/accuracy_7_of_7‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 8"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.464457243680954 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1312144696712494 (particle/f_score‚0.41921937465667725 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3399471044540405 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974355459213257 jet/accuracy_5_of_6‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.47200462222099304 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3684210479259491 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 8"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.348710298538208 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3754992187023163 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.19443166255950928 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.13534174859523773 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3995591998100281 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.29642099142074585 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ,jet/accuracy_4_of_5‚0.002481389557942748 particle/accuracy_5_of_5‚1 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.38927754759788513 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36314472556114197 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09264875203371048 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4304564893245697 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.1560983806848526 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.48469242453575134 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.05592592805624008 ,jet/accuracy_4_of_4‚0.003168202703818679 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.02850494347512722 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3710178732872009 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12471263855695724 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.49316784739494324 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12019733339548111 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23172755539417267 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.4122837781906128 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9961372017860413 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.14853380620479584 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.14140497148036957 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8564716577529907 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 +"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.25 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.44809868931770325 particle/accuracy_3_of_5‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 8 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.02380952425301075 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0357142873108387 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4634985327720642 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ( Purity/3FRt1SRqqt2FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5175719261169434 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.4340464472770691 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.47813814878463745 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.19255951046943665 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.14305555820465088 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2777777910232544 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2348484843969345 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.6904762387275696 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2506897449493408 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4515853822231293 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09157183766365051 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1941763460636139 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 jet/accuracy_6_of_6‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.14334282279014587 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4601907432079315 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9928579926490784 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9222223162651062 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.40833333134651184 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1386958211660385 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9892777800559998 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48571476340293884 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9955452680587769 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4336555004119873 particle/accuracy_2_of_3‚1 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.390625 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6790979504585266 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4377272427082062 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.15045858919620514 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2998954951763153 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9900306463241577 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40804848074913025 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18987415730953217 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.47004571557044983 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02648942731320858 particle/accuracy_5_of_7‚1 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4610220491886139 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.31440478563308716 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.2076190561056137 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 particle/accuracy_1_of_6‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 % _timestamp‚1.7883937090697289e+09 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9892541766166687 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.993921160697937 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.48066726326942444 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.12404762953519821 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.45030680298805237 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.19901201128959656 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19477839767932892 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5777778029441833 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.023403571918606758 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.06328042596578598 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.40743696689605713 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9934173822402954 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.061511993408203125 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.44136562943458557 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3532727360725403 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_2_of_5‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.46666663885116577 _step‚391 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.16744762659072876 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.375 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 particle/accuracy_1_of_7‚1 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.29979708790779114 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3130760192871094 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11342593282461166 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 trainer/global_step‚4831 :"Purity/2FRt0SRqqt0FBt/event_purity‚0.01506772730499506 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3698415458202362 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9747023582458496 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9913756847381592 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8991770148277283 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4519194960594177 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3296296298503876 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.27629798650741577 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1553741991519928 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.21054469048976898 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4653668701648712 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 particle/accuracy_4_of_7‚1 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.19585253298282623 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2694639563560486 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_7‚1 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22186264395713806 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.445833295583725 +jet/accuracy_4_of_6‚0.07894736528396606 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07970396429300308 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.992557168006897 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9166666865348816 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_1‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9866006374359131 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.18666668236255646 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.09283608198165894 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44688478112220764 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9898256659507751 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41729649901390076 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4095747470855713 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.026581183075904846 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3408532440662384 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 )jet/accuracy_1_of_1‚0.456368625164032 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.19266614317893982 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *particle/specificity‚0.953597366809845 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42856690287590027 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.44394436478614807 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 epoch‚7 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1612892895936966 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.25099843740463257 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 *jet/accuracy_2_of_6‚0.4298245310783386 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.10255540162324905 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_6_of_6‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 _runtime‚4464.827933676 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3668217360973358 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4466589689254761 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5030864477157593 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_propog.ø^rtion‚0.04731602966785431 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.4325714111328125 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.41050171852111816 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8267597556114197 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.20019203424453735 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.11948460340499878 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3076923191547394 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3571428656578064 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8846477270126343 particle/accuracy_2_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.42185521125793457 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.42171716690063477 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.47101452946662903 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.20725905895233154 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3706369996070862 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9811885356903076 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.350166380405426 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 )jet/accuracy_2_of_4‚0.346370130777359 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.21399180591106415 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.22111640870571136 particle/accuracy_1_of_5‚1 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 particle/accuracy_4_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13424888253211975 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.005971279926598072 particle/accuracy_2_of_2‚1 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4542991816997528 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_2_of_4‚1 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_4‚0.05535747855901718 particle/accuracy_1_of_3‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19455312192440033 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.45026135444641113 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1568627655506134 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1619195193052292 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.47271162271499634 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2788989841938019 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.21807622909545898 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4384588301181793 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.006451612804085016 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.14814814925193787 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02688361145555973 particle/accuracy_3_of_3‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 jet/accuracy_4_of_7‚0 8 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.02380952425301075 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4173242449760437 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9255732893943787 particle/accuracy_1_of_2‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.23427610099315643 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.13995949923992157 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.2282467484474182 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.003225806402042508 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06913421303033829 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.42732468247413635‡Òi|®’é‚(:‡ òñâÔ£è¯u gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚141.488' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.2976 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0„‚ŠŠKê‚(:À òñâÔ¼¬Áy disk.sda3.out‚ 1237.0078125" proc.memory.availableMB‚1292.5% disk./.usageGB‚1065.8858070373535' disk.nvme0c0n1.out‚48028.9033203125# proc.memory.rssMB‚ 2892.85546875 proc.cpu.threads‚17 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0 network.recv‚ 1.556005e+06 disk.nvme0n1.in‚ 265.54296875 disk.rbd0.in‚0 disk.sda2.out‚0 disk.dm-1.in‚0.125 disk.rbd1.in‚0 network.sent‚ 1.0073549e+07 disk.sda3.in‚0.125 disk.rbd2.in‚0# memory_percent‚92.1112060546875* proc.memory.percent‚17.656588554382324 disk.dm-1.out‚ 1237.0078125 disk.rbd1.out‚ 2.3896484375 disk.drbd1777.out‚0) disk./.usagePercent‚71.53249084803785% disk.nvme0n1.out‚48028.9033203125 disk.sda.in‚0.125 disk.sda.out‚ 1237.0078125 disk.sda1.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 2.349609375" disk.nvme0c0n1.in‚ 265.54296875! disk.rbd2.out‚387.62255859375 disk.sda2.in‚0 disk.dm-0.in‚ 265.54296875" disk.dm-0.out‚48028.9033203125 disk.sda1.out‚0 cpu‚25.65689323452735Z» ÷2ë- _wandbruntime‚4482  _runtime‚4482-`-üÈì ' lr-AdamW/pg1‚0.00019564225757915982 ' lr-AdamW/pg2‚0.00019564225757915982 trainer/global_step‚4849 $ _timestamp‚1.788393726704415e+09 _runtime‚4482.460389743 _step‚392jk]¦ÍíÇ ' lr-AdamW/pg2‚0.00019564225757915982 trainer/global_step‚4849 $ _timestamp‚1.788393726704415e+09 _runtime‚4482.460389743 _step‚392 ' lr-AdamW/pg1‚0.00019564225757915982ˆÆ"¡(2î- _wandbruntime‚4482  _runtime‚4482AqPãì'ïæO UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3980610436968 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚83290980 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚51056319 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚83290980 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 186828966 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 trainer/global_step‚4849 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26095914 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚326919936529408 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3980802433508 _step‚393 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 % _timestamp‚1.7883937267054386e+09 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 186806366 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚330922838388736 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚330922838388736 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚333759266213376 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚329554973971968 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4204292241408 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚322937267205992 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚326919919490048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4204532270080 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚326919919490048 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4003115077632 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚83290982 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚326919936529408 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚322937476241892 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚318956673808384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 103515386 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 103515386 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚77152230 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚533 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚329554998875648 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚330923051607040 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4002918898688 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚83290982 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚77152766 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚4482.461638386 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4002918898688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 103537984 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 103537984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚51056316 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750®ºNíñ'ðëO TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 % _timestamp‚1.7883937267054386e+09 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 186806366 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚330922838388736 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚330922838388736 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚333759266213376 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚329554973971968 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4204292241408 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚322937267205992 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚326919919490048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚536 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4204532270080 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚326919919490048 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4003115077632 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚83290982 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚326919936529408 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚322937476241892 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚318956673808384 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 103515386 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 103515386 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚77152230 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚533 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚329554998875648 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚330923051607040 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4002918898688 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚83290982 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚77152766 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚4482.461638386 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4002918898688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 103537984 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 103537984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚51056316 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3980610436968 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚83290980 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚51056319 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚83290980 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 186828966 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 trainer/global_step‚4849 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26095914 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚326919936529408 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3980802433508 _step‚393 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0‰ŽYó2ñ- _wandbruntime‚4483  _runtime‚4483_”d1 òïM D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 $ _timestamp‚1.788393727477254e+09 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚326984975566336 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚330988889048064 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4003717289472 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚77167399 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚319020256153600 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 186866384 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 REDeviceStatsMonitor.I ïÄon_train_batch_end/inactive_split.large_pool.freed‚51066350 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚330988889048064 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚77167901 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚26101049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚83307500 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26101548 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 103558882 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 103558882 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 103536258 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚333825894558720 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256530432 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3981596316664 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4003717289472 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4003913481728 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚4849 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚323001869509624 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚83307502 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚83307500 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 _runtime‚4483.234087254 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚323001660473516@ UÉú&óôM H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 186843758 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚329620499468288 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚326984958526976 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚329620524371968 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚83307502 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4205138560000 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3981404319916 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 103536258 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚326984975566336 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚330988889048064 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4003717289472 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚77167399 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚319020256153600 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 186866384 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚51066350 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚330988889048064 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚77167901 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚26101049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚83307500 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26101548 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 103558882 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 103558882 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 103536258 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚333825894558720 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 256530432 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3981596316664 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4003717289472 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4003913481728 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚4849 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚323001869509624 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚83307502 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚83307500 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 _runtime‚4483.234087254 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚323001660473516 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 $ _timestamp‚1.788393727477254e+09 PDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788393769698151e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚334214737602560 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚330171888375808 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚84116982 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚509 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 252349440 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚326156957754200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚84116980 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 104584936 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 104562338 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 188701918 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚337077952989184 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 _step‚397 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4042866266112 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4246704201216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4043062445056 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚334214950820864 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚334214737Õ‰›¢Ë602560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚322136643037184 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚77910617 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚51558019 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚332831248787968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4246931646976 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚26352598 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 188679318 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚84116982 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 104584936ýíý*ð'‚êO TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚84116980 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚77910105 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚322136625997824 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4020331756376 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚330171888375808 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚84116982 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚509 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 252349440 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚326156957754200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚84116980 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 104584936 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227445760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 104562338 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 188701918 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚337077952989184 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 _step‚397 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4042866266112 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4246704201216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4043062445056 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚334214950820864 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚334214737602560 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚322136643037184 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚77910617 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚51558019 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚332831248787968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4246931646976 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚26352598 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 188679318 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚84116982 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 104584936 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4043062445056 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚332831273691648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚330171871336448 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4020523752916 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚330171888375808 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚51558016 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788393769698151e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚334214737602560 JDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚77925222 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚332896799187968 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚334280788261888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚326221560057832 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4043664656896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚334280575030272 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 _step‚398 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 188739336 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚84133500 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚330236927412736 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚51568053 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 188739336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _runtime‚4526.223511823 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231626752 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 104583210 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚330236927412736 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 188716710 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚84133502 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4043860849152 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚322200242421760 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚332896774284288 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚77925752 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚4899 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4043664656896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚84133500 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26357699 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 104583210 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚330236910373376 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚51568050 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚26357172 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4021317636072 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚84133502)Pe]û&…õM UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4043664656896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚334280575030272 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 _step‚398 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 188739336 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚84133500 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚330236927412736 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚51568053 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚527 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 188739336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _runtime‚4526.223511823 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 231626752 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 104583210 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚330236927412736 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 188716710 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚84133502 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4043860849152 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚322200242421760 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚332896774284288 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚77925752 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 trainer/global_step‚4899 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4043664656896 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚84133500 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26357699 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 104583210 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚330236910373376 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚51568050 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚26357172 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4021317636072 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚84133502 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 104605834 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚77925222 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚332896799187968 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚334280788261888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚326221560057832Ž~EN2†- _wandbruntime‚4526  _runtime‚4526,Tá¿‡Ï 2loss/SRqqt4/assignment_loss‚1.9892357924240238 0loss/FRt3/assignment_loss‚2.6621187104695085 1loss/SRqqt3/detection_loss‚1.7481525705920324 -loss/FBt1/detection_loss‚2cû–.78869735399882 epoch‚8 1loss/SRqqt2/assignment_loss‚0.933441719257048 1loss/FBt2/assignment_loss‚0.15300904531079373 /loss/FRt2/detection_loss‚1.5968769878225844 1loss/SRqqt1/detection_loss‚1.7874522695690394 .loss/FBt2/detection_loss‚0.595988547461192 0loss/FRt2/assignment_loss‚1.5012642593727898 1loss/SRqqt1/assignment_loss‚1.665567664001137 0loss/FBt1/assignment_loss‚0.1962780279000678 0loss/FBt4/assignment_loss‚0.0450884892726952 0loss/SRqqt4/detection_loss‚2.148247292012344 /loss/FBt3/detection_loss‚1.9765481623736294 /loss/FRt4/assignment_loss‚3.353805339181578 0loss/FBt3/assignment_loss‚0.3809034664591722 /loss/FRt1/detection_loss‚2.5540718339480364 .loss/FBt4/detection_loss‚3.011554013599049 %loss/total_loss‚41.95765038204938 .loss/FRt3/detection_loss‚2.598150534042414 1loss/SRqqt2/detection_loss‚1.0898941194253235 trainer/global_step‚4899 % _timestamp‚1.7883937704688942e+09 .loss/FRt4/detection_loss‚2.595180283119152 _runtime‚4526.224682139 /loss/FRt1/assignment_loss‚3.013969937702278 2loss/SRqqt3/assignment_loss‚1.5721539627346728 _step‚399hyJ]ZˆÔ 0loss/FRt3/assignment_loss‚2.6621187104695085 1loss/SRqqt3/detection_loss‚1.7481525705920324 -loss/FBt1/detection_loss‚2.78869735399882 epoch‚8 1loss/SRqqt2/assignment_loss‚0.933441719257048 1loss/FBt2/assignment_loss‚0.15300904531079373 /loss/FRt2/detection_loss‚1.5968769878225844 1loss/SRqqt1/detection_loss‚1.7874522695690394 .loss/FBt2/detection_loss‚0.595988547461192 0loss/FRt2/assignment_loss‚1.5012642593727898 1loss/SRqqt1/assignment_loss‚1.665567664001137 0loss/FBt1/assignment_loss‚0.1962780279000678 0loss/FBt4/assignment_loss‚0.0450884892726952 0loss/SRqqt4/detection_loss‚2.148247292012344 /loss/FBt3/detection_loss‚1.9765481623736294 /loss/FRt4/assignment_loss‚3.353805339181578 0loss/FBt3/assignment_loss‚0.3809034664591722 /loss/FRt1/detection_loss‚2.5540718339480364 .loss/FBt4/detection_loss‚3.011554013599049 %loss/total_loss‚41.95765038204938 .loss/FRt3/detection_loss‚2.598150534042414 1loss/SRqqt2/detection_loss‚1.0898941194253235 trainer/global_step‚4899 % _timestamp‚1.7883937704688942e+09 .loss/FRt4/detection_loss‚2.595180283119152 _runtime‚4526.224682139 /loss/FRt1/assignment_loss‚3.013969937702278 2loss/SRqqt3/assignment_loss‚1.5721539627346728 _step‚399 2loss/SRqqt4/assignment_loss‚1.9892357924240238Û`)¬’‰‚(:‡ ®òâÔ‰¤Îu gpu.0.gpu‚21.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.083' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.6166 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.¤{MŠ‚(: ®òâÔÏóÄx! disk.rbd2.out‚390.91552734375 disk.nvme0n1.in‚ 265.54296875 disk.sda.out‚ 1284.390625* proc.memory.percent‚17.656588554382324 network.sent‚ 1.0275095e+07% disk./.usageGB‚1065.8848342895508 disk.drbd1777.out‚0" disk.nvme0c0n1.in‚ 265.54296875' disk.nvme0c0n1.out‚48468.1435546875 disk.rbd2.in‚0 disk.sda.in‚0.125 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.sda3.in‚0.125 disk.sda3.out‚ 1284.390625 disk.dm-1.out‚ 1284.390625 disk.rbd1.in‚0 disk.rbd0.out‚ 0.2978515625 network.recv‚ 1.575436e+06 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 265.54296875" disk.dm-0.out‚48468.1435546875 disk.rbd1.out‚2.515625 disk.rbd0.in‚0 disk.dm-1.in‚0.125$ memory_percent‚92.11461544036865 proc.cpu.threads‚17 disk.sda1.out‚0) disk./.usagePercent‚71.53242556611663 disk.sda1.in‚0 disk.sda2.out‚0# proc.memory.rssMB‚ 2892.85546875 cpu‚25.656653300765633 disk.sda2.in‚0% disk.nvme0n1.out‚48468.1435546875) proc.memory.availableMB‚ 1291.94140625¦ œ‹‚(:‘ ½òâÔÔö¡u gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.84' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.567999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚05VŒ‚(:Ë ½òâÔùÑÅw) disk./.usagePercent‚71.53242940622965! disk.rbd2.out‚392.59521484375# proc.memory.rssMB‚ 2892.85546875 disk.rbd3.in‚0 disk.dm-1.in‚0.125 network.sent‚ 1.0361767e+07 network.recv‚ 1.580588e+06 disk.sda3.in‚0.125" disk.nvme0c0n1.in‚ 265.54296875( disk.nvme0c0n1.out‚48469.02001953125* proc.memory.percent‚17.656588554382324 disk.sda3.out‚ 1300.01171875 disk.dm-1.out‚ 1300.01171875& disk.nvme0n1.out‚48469.02001953125 disk.rbd0.in‚0) proc.memory.availableMB‚ 1291.44921875% disk./.usageGB‚1065.8848915100098 disk.rbd3.out‚ 2.349609375 disk.rbd1.out‚2.515625 disk.rbd2.in‚0 disk.sda.in‚0.125 disk.sda1.out‚0# disk.dm-0.out‚48469.02001953125 disk.nvme0n1.in‚ 265.54296875 disk.sda.out‚ 1300.01171875$ memory_percent‚92.11761951446533 disk.dm-0.in‚ 265.54296875 disk.rbd1.in‚0 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.sda2.out‚0 cpu‚25.656643363065935 disk.drbd1777.in‚0 disk.drbd1777.out‚0 proc.cpu.threads‚178–g‚(:… ÌòâÔ«öÎu gpu.0.gpu‚25.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚91.295' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.259 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0hg<IŽ‚(:¾ ÌòâÔû׈y disk.rbd2.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0# disk.nvme0n1.out‚48567.33203125 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 1315.11328125# proc.memory.rssMB‚ 2892.85546875* proc.memory.percent‚17.656588554382324 cpu‚25.656765284295997 network.sent‚ 1.0368023e+07) disk./.usagePercent‚71.53243094227484! disk.rbd2.out‚392.59521484375 disk.sda.in‚0.125" disk.nvme0c0n1.in‚ 265.54296875 disk.sda3.in‚0.125% proc.memory.availableMB‚ 1291.6875 network.recv‚ 1.584919e+06 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-0.out‚48567.33203125 disk.dm-1.out‚ 1315.11328125 proc.cpu.threads‚17% disk./.usageGB‚1065.8849143981934 disk.rbd1.out‚2.515625% disk.nvme0c0n1.out‚48567.33203125 disk.dm-1.in‚0.125 disk.sda1.out‚0 disk.rbd1.in‚0 disk.sda.out‚ 1315.11328125 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375$ memory_percent‚92.11616516113281 disk.nvme0n1.in‚ 265.54296875 disk.dm-0.in‚ 265.54296875 disk.sda1.in‚0Ô›/è2- _wandbruntime‚4568  _runtime‚45688˜ÒèÈ $ _timestamp‚1.788393812866603e+09 _runtime‚4568.622600719 _step‚400 ' lr-AdamW/pg1‚0.00016522178044834875 ' lr-AdamW/pg2‚0.00016522178044834875 trainer/global_step‚4949‹WçÍ‘Ç $ _timestamp‚1.788393812866603e+09 _runtime‚4568.622600719 _step‚400 ' lr-AdamW/pg1‚0.00016522178044834875 ' lr-AdamW/pg2‚0.00016522178044834875 trainer/global_step‚4949+³Ü2’- _wandbruntime‚4568  _runtime‚4568„!pòì'“æO QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 105609290 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 trainer/global_step‚4949 _step‚401 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 190552270 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚329376857338308 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚329376648302408 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚333423840222208 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 105631888 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚337506636816384 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚336107548507648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚340396868472320 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4060245072324 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚337506850034688 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚52059719 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚78669719 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚333423823182848 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚340396607734272 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4060053075784 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚336107523603968 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 105631888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26610003 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚337506636816384 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚325316612265984 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚52059716 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚524 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚4568.623977725 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚84942980 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4083009812480 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚78670246 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚78669719 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚333423823182848 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚340396607734272 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4060053075784 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚336107523603968 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 105631888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26610003 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚337506636816384 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚325316612265984 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚52059716 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚524 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚4568.623977725 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚84942980 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4083009812480 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚78670246 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚329376857338308 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚329376648302408 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚333423840222208 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 105631888 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚337506636816384 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚336107548507648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚340396868472320 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4060245072324 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0‘yÖ2•- _wandbruntime‚4569  _runtime‚4569·¦d­î–ïM L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 190612288 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚336173049100288 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚336173074003968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 190612288 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚537 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26615619 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _runtime‚4569.407408445 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4083808216576 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚329441250606040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 190589662 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚78684832 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚52069753 RDDeviceS<ÛuätatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚84959500 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚337572687475712 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚52069750 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚340462962248192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4290157357568 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚325380211650560 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 105652786 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚333488862219776 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚540 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _step‚402 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4083612024320 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4083612024320 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 105652786 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚329441041569932 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚84959502 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 $ _timestamp‚1.788393813650887e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4083808216576]ó¨‹ú&—ôM H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚540 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 _step‚402 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4083612024320 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4083612024320 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 105652786 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚329441041569932 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚84959502 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 $ _timestamp‚1.788393813650887e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4083808216576 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 190612288 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚336173049100288 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚336173074003968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 190612288 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚537 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26615619 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _runtime‚4569.407408445 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4083808216576 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚329441250606040 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 190589662 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚78684832 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚52069753 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚84959500 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚337572687475712 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚52069750 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚340462962248192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4290157357568 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚325380211650560 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 105652786 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚333488862219776 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936’‚šx2˜- _wandbruntime‚4569  _runtime‚4569ûå-mX™Ò /loss/FBt1/detection_loss‚2.4648167771451615 _runtime‚4569.408712734 0loss/SRqqt3/assignment_loss‚1.44448116547183 /loss/FRt3/assignment_loss‚2.924621239830466 /loss/FRt3/detection_loss‚2.4352223066722645 0loss/SRqqt2/detection_loss‚0.937515533594198 .loss/FBt4/detection_loss‚3.100768582387404 /loss/FRt1/assignment_loss‚2.858472240633435 /loss/FRt2/detection_loss‚1.6037659948772711 1loss/SRqqt1/detection_loss‚1.6639316248543121 /loss/FBt2/detection_loss‚0.5598881100332993 /loss/FRt2/assignment_loss‚1.459680200918861 /loss/FRt4/assignment_loss‚3.834015174044503 1loss/SRqqt2/assignment_loss‚0.982067866001798 1loss/FBt3/assignment_loss‚0.29686780575788624 1loss/FBt4/assignment_loss‚0.25199808449013456 epoch‚8 trainer/global_step‚4949 1loss/SRqqt1/assignment_loss‚1.783111116684535 2loss/SRqqt4/assignment_loss‚1.9693628500270492 1loss/FBt2/assignment_loss‚0.10943446352305432 .loss/FRt1/detection_loss‚2.312518555753761 /loss/FRt4/detection_loss‚2.6046603752507105 &loss/total_loss‚42.140193348632906 % _timestamp‚1.7883938136527452e+09 1loss/SRqqt3/detection_loss‚1.8954023110239129 _step‚403 .loss/FBt3/detection_loss‚2.304708574499403 2loss/FBt1/assignment_loss‚0.034269546251701234 0loss/SRqqt4/detection_loss‚2.308612848905956'nýD]š× /loss/FRt1/assignment_loss‚2.858472240633435 /loss/FRt2/detection_loss‚1.6037659948772711 1loss/SRqqt1/detection_loss‚1.6639316248543121 /loss/FBt2/detection_loss‚0.5598881100332993 /loss/FRt2/assignment_loss‚1.459680200918861 /loss/FRt4/assignment_loss‚3.834015174044503 1loss/SRqqt2/assignment_loss‚0.982067866001798 1loss/FBt3/assignment_loss‚0.29686780575788624 1loss/FBt4/assignment_loss‚0.25199808449013456 epoch‚8 trainer/global_step‚4949 1loss/SRqqt1/assignment_loss‚1.783111116684535 2loss/SRqqt4/assignment_loss‚1.9693628500270492 1loss/FBt2/assignment_loss‚0.10943446352305432 .loss/FRt1/detection_loss‚2.312518555753761 /loss/FRt4/detection_loss‚2.6046603752507105 &loss/total_loss‚42.140193348632906 % _timestamp‚1.7883938136527452e+09 1loss/SRqqt3/detection_loss‚1.8954023110239129 _step‚403 .loss/FBt3/detection_loss‚2.304708574499403 2loss/FBt1/assignment_loss‚0.034269546251701234 0loss/SRqqt4/detection_loss‚2.308612848905956 /loss/FBt1/detection_loss‚2.4648167771451615 _runtime‚4569.408712734 0loss/SRqqt3/assignment_loss‚1.44448116547183 /loss/FRt3/assignment_loss‚2.924621239830466 /loss/FRt3/detection_loss‚2.4352223066722645 0loss/SRqqt2/detection_loss‚0.937515533594198 .loss/FBt4/detection_loss‚3.100768582387404“÷/H(:’ ÛòâÔ¶—»u gpu.0.gpu‚35.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.432' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.486399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚02y-Kœ‚(:À ÛòâÔêäÞx disk.sda.in‚0.125' disk.nvme0c0n1.out‚48666.0732421875 disk.dm-0.in‚ 265.54296875* proc.memory.percent‚17.656588554382324% disk./.usageGB‚1065.8849143981934 disk.sda3.out‚ 1315.828125% disk.nvme0n1.out‚48666.0732421875 disk.sda.out‚ 1315.828125 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.dm-1.out‚ 1315.828125" disk.dm-0.out‚48666.0732421875 network.sent‚ 1.0374535e+07 disk.nvme0n1.in‚ 265.54296875 disk.rbd0.out‚ 0.2978515625$ memory_percent‚92.11688041687012 cpu‚25.65664049677861 disk.dm-1.in‚0.125 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 2.349609375 disk.sda2.out‚0! disk.rbd2.out‚394.24755859375 disk.drbd1777.out‚0( proc.memory.availableMB‚ 1291.5703125# proc.memory.rssMB‚ 2892.85546875 network.recv‚ 1.589555e+06 disk.sda3.in‚0.125 disk.rbd1.out‚2.515625 disk.rbd0.in‚0 disk.drbd1777.in‚0" disk.nvme0c0n1.in‚ 265.54296875 proc.cpu.threads‚17) disk./.usagePercent‚71.53243094227484ÓTûÍ’‚(:‡ êòâÔÆÂu gpu.0.gpu‚57.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.791' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.1582 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0j¦Qž‚(:Æ êòâÔ¬»Õx disk.sda1.out‚0' disk.nvme0c0n1.out‚48666.3701171875* proc.memory.percent‚17.656588554382324 disk.sda2.in‚0) disk./.usagePercent‚71.53242966223718$ disk./.usageGB‚1065.884895324707 disk.rbd2.in‚0 disk.sda1.in‚0 disk.rbd3.out‚ 2.349609375$ memory_percent‚92.11688041687012( proc.memory.availableMB‚ 1291.5703125 network.sent‚ 1.0461326e+07 disk.sda3.out‚ 1317.01171875 disk.rbd1.out‚2.515625 disk.sda.out‚ 1317.01171875 disk.drbd1777.out‚0 disk.rbd3.in‚0# proc.memory.rssMB‚ 2892.85546875 disk.rbd0.in‚0 cpu‚25.656513543198272 proc.cpu.threads‚17 disk.dm-0.in‚ 265.54296875 disk.rbd0.out‚ 0.2978515625" disk.nvme0c0n1.in‚ 265.54296875 network.recv‚ 1.595206e+06 disk.sda3.in‚0.125 disk.dm-1.in‚0.125 disk.nvme0n1.in‚ 265.54296875 disk.dm-1.out‚ 1317.01171875 disk.rbd1.in‚0! disk.rbd2.out‚394.27099609375% disk.nvme0n1.out‚48666.3701171875" disk.dm-0.out‚48666.3701171875 disk.sda.in‚0.125 disk.drbd1777.in‚0 disk.sda2.out‚0W><œ’Ÿ‚(:‡ ùòâÔ¦Èu gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚144.896' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.9792 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¬ÅZ G ‚(:¼ ùòâÔ£Îëx disk.sda2.in‚0 disk.dm-1.in‚0.125 disk.dm-1.out‚ 1317.546875 disk.rbd1.out‚2.515625 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 proc.cpu.threads‚17) disk./.usagePercent‚71.53243273432759 disk.rbd3.out‚ 2.349609375% disk./.usageGB‚1065.8849411010742 disk.sda3.in‚0.125 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.54296875& disk.nvme0c0n1.out‚48764.833984375) proc.memory.availableMB‚ 1291.56640625* proc.memory.percent‚17.656588554382324 disk.sda2.out‚0 disk.sda3.out‚ 1317.546875 disk.drbd1777.out‚0" disk.nvme0c0n1.in‚ 265.54296875 network.sent‚ 1.046684e+07 network.recv‚ 1.59892e+06 disk.sda.in‚0.125 disk.drbd1777.in‚0 disk.dm-0.in‚ 265.54296875 disk.sda1.out‚0 cpu‚25.65654396568412 disk.rbd1.in‚0! disk.rbd2.out‚395.98583984375 disk.sda.out‚ 1317.546875 disk.sda1.in‚0$ memory_percent‚92.11690425872803# proc.memory.rssMB‚ 2892.85546875 disk.rbd3.in‚0! disk.dm-0.out‚48764.833984375$ disk.nvme0n1.out‚48764.833984375Oð ª2¡- _wandbruntime‚4611  _runtime‚4611h(húɢà ' lr-AdamW/pg1‚0.00015087456644508578 ' lr-AdamW/pg2‚0.00015087456644508578 trainer/global_step‚4999 % _timestamp‚1.7883938558114958e+09 _runtime‚4611.567479227 _step‚404¢ ³!Î£È ' lr-AdamW/pg1‚0.00015087456644508578 ' lr-AdamW/pg2‚0.00015087456644508578 trainer/global_step‚4999 % _timestamp‚1.7883938558114958e+09 _runtime‚4611.567479227 _step‚404”Ói§u2¤- _wandbruntime‚4611  _runtime‚4611ü²ª %¥æO F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 192425222 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚340798536030208 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4122761000960 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚4999 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26868881 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 192447822 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 106656242 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4122957179904 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4122761000960 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4099966391732 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚79430812 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚336675775029248 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _runtime‚4611.572003555 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚340798749248512 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚79430297 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚52561416 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 106678840 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚340798749248512 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚339383798419968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4331501673984 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚85768980 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚336675775029248 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚26869393 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚328496564455424 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚328496581494784 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚336675792068608 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4331267936768 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚332596338850616 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚340798536030208 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4122761000960 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚4999 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚26868881 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 192447822 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 106656242 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4122957179904 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4122761000960 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4099966391732 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚79430812 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚336675775029248 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _runtime‚4611.572003555 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚340798749248512 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚79430297 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚52561416 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 106678840 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚340798749248512 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚339383798419968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4331501673984 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚85768980 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚336675775029248 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚26869393 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚328496564455424 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚328496581494784 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚336675792068608 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚512 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4331267936768 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚332596338850616 J6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚52571450 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚340864586689536 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚85785502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚336740814066176 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 106699738 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚52571453 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4123559391744 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚336740831105536 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4123755584000 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26874705 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 192462614 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚340864586689536 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚343781429062656 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4332345161728 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4100568278140 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883938565918136e+09 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 106699738 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚79445633 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚85785502 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4123559391744 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 192485240 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4100760274888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚328560163840000 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 trainer/global_step‚4999 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚340864373457920 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚332660732118140 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚332660941154248 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3Ä9Lû&©õM [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚336740831105536 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 106677114 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚525 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 192462614 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚52571450 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚340864586689536 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚85785502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚336740814066176 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 106699738 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚52571453 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4123559391744 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚336740831105536 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4123755584000 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚26874705 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 192462614 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚340864586689536 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚343781429062656 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4332345161728 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4100568278140 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883938565918136e+09 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 106699738 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚79445633 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚85785502 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4123559391744 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 192485240 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4100760274888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚328560163840000 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 trainer/global_step‚4999 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚340864373457920 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚332660732118140 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚332660941154248 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚339449323916288 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 _step‚406 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚79446158 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚85785500 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚343781693981696 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824–UöÒw2ª- _wandbruntime‚4612  _runtime‚4612ønÉ\«Ö _runtime‚4612.349278676 2loss/SRqqt1/assignment_loss‚1.4258107943590905 1loss/SRqqt3/detection_loss‚1.7016759241882125 0loss/FBt2/detection_loss‚0.42650011109071784 /loss/FBt4/detection_loss‚2.9037809239493475 2loss/SRqqt2/assignment_loss‚0.9293678007288171 1loss/FBt2/assignment_loss‚0.11060625320621108 /loss/FRt3/detection_loss‚2.3894583118218247 epoch‚8 0loss/SRqqt4/detection_loss‚2.762523033345739 .loss/FRt2/detection_loss‚1.758336283026203 /loss/FRt3/assignment_loss‚3.151162383376911 0loss/FBt1/assignment_loss‚0.2642272213072699 2loss/FBt4/assignment_loss‚0.030852693977067247 _step‚407 0loss/FRt1/assignment_loss‚2.8645990913914097 /loss/FRt2/assignment_loss‚1.565793198923911 /loss/FRt4/detection_loss‚2.5901308666776726 1loss/SRqqt2/detection_loss‚0.9209998002275824 .loss/FBt1/detection_loss‚2.138144163148744 0loss/FBt3/assignment_loss‚0.4587276105557976 .loss/FBt3/detection_loss‚4.212615765631199 % _timestamp‚1.7883938565934875e+09 1loss/SRqqt1/detection_loss‚1.7614995461897316 trainer/global_step‚4999 &loss/total_loss‚44.255746263228055 /loss/FRt4/assignment_loss‚3.970101718549375 2loss/SRqqt3/assignment_loss‚1.6717890469259338 1loss/SRqqt4/assignment_loss‚1.907206310192123 /loss/FRt1/detection_loss‚2.3398374104371635˜-ÖûC¬Û 2loss/SRqqt2/assignment_loss‚0.9293678007288171 1loss/FBt2/assignment_loss‚0.11060625320621108 /loss/FRt3/detection_loss‚2.3894583118218247 epoch‚8 0loss/SRqqt4/detection_loss‚2.762523033345739 .loss/FRt2/detection_loss‚1.758336283026203 /loss/FRt3/assignment_loss‚3.151162383376911 0l]~J¸oss/FBt1/assignment_loss‚0.2642272213072699 2loss/FBt4/assignment_loss‚0.030852693977067247 _step‚407 0loss/FRt1/assignment_loss‚2.8645990913914097 /loss/FRt2/assignment_loss‚1.565793198923911 /loss/FRt4/detection_loss‚2.5901308666776726 1loss/SRqqt2/detection_loss‚0.9209998002275824 .loss/FBt1/detection_loss‚2.138144163148744 0loss/FBt3/assignment_loss‚0.4587276105557976 .loss/FBt3/detection_loss‚4.212615765631199 % _timestamp‚1.7883938565934875e+09 1loss/SRqqt1/detection_loss‚1.7614995461897316 trainer/global_step‚4999 &loss/total_loss‚44.255746263228055 /loss/FRt4/assignment_loss‚3.970101718549375 2loss/SRqqt3/assignment_loss‚1.6717890469259338 1loss/SRqqt4/assignment_loss‚1.907206310192123 /loss/FRt1/detection_loss‚2.3398374104371635 _runtime‚4612.349278676 2loss/SRqqt1/assignment_loss‚1.4258107943590905 1loss/SRqqt3/detection_loss‚1.7016759241882125 0loss/FBt2/detection_loss‚0.42650011109071784 /loss/FBt4/detection_loss‚2.9037809239493475—aG­‚(:’ ˆóâÔ‰›¼u gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.831' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.766199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ïÀ6T®‚(:É ˆóâÔ´Éx disk.sda2.out‚0# disk.dm-0.out‚48765.20458984375 disk.rbd2.in‚0 disk.drbd1777.in‚0) disk./.usagePercent‚71.53241558182279& disk.nvme0n1.out‚48765.20458984375" disk.nvme0c0n1.in‚ 265.54296875 disk.sda2.in‚0 disk.rbd3.out‚ 2.349609375 disk.sda3.in‚0.125 disk.sda3.out‚ 1318.13671875 disk.dm-1.in‚0.125 disk.rbd1.out‚2.515625! disk.rbd2.out‚397.52490234375 disk.nvme0n1.in‚ 265.54296875 disk.sda.out‚ 1318.13671875 disk.rbd0.out‚ 0.2978515625 disk.rbd3.in‚0) proc.memory.availableMB‚ 1291.00390625# proc.memory.rssMB‚ 2892.85546875% disk./.usageGB‚1065.8846855163574 disk.drbd1777.out‚0 disk.dm-0.in‚ 265.54296875$ memory_percent‚92.12033748626709* proc.memory.percent‚17.656588554382324 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.sda.in‚0.125 cpu‚25.65647821087018( disk.nvme0c0n1.out‚48765.20458984375 disk.dm-1.out‚ 1318.13671875 disk.sda1.out‚0 disk.rbd0.in‚0 network.sent‚ 1.0472953e+07 network.recv‚ 1.60357e+06 disk.sda1.in‚0nĵöœ¯‚(:‘ —óâÔ°Õ˜u gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.08' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.216000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ã«ÓA°‚(:¶ —óâÔ´Ì®x disk.drbd1777.out‚0 disk.sda2.in‚0! disk.dm-0.out‚48863.662109375 disk.rbd1.in‚0! disk.rbd2.out‚397.66552734375 disk.dm-1.in‚0.125 disk.rbd1.out‚2.515625* proc.memory.percent‚17.656588554382324 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 265.6328125 disk.dm-0.in‚ 265.6328125$ memory_percent‚92.11893081665039 network.recv‚ 1.610293e+06 disk.sda1.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 1319.78125) disk./.usagePercent‚71.53241558182279 disk.rbd3.out‚ 2.349609375% disk./.usageGB‚1065.8846855163574$ disk.nvme0n1.out‚48863.662109375 disk.sda1.out‚0' proc.memory.availableMB‚ 1291.234375 disk.drbd1777.in‚0# proc.memory.rssMB‚ 2892.85546875 network.sent‚ 1.056076e+07 disk.sda.in‚0.125 disk.rbd0.in‚0& disk.nvme0c0n1.out‚48863.662109375 disk.rbd3.in‚0 disk.sda3.in‚0.125 disk.dm-1.out‚ 1319.78125 cpu‚25.656507959703212 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda.out‚ 1319.78125 disk.rbd0.out‚ 0.2978515625rV†Ù‘±‚(:† ¦óâÔ±›Äu gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.0248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Š;ÇI²‚(:¾ ¦óâÔ€õÿx disk.dm-1.in‚0.125 disk.dm-1.out‚ 1337.09375& disk.nvme0n1.out‚48962.01123046875 disk.sda.out‚ 1337.09375 disk.drbd1777.out‚0) proc.memory.availableMB‚ 1291.48046875 disk.sda2.out‚0# disk.dm-0.out‚48962.01123046875 disk.rbd1.in‚0 disk.nvme0n1.in‚ 265.6328125 network.sent‚ 1.056611e+07( disk.nvme0c0n1.out‚48962.01123046875 disk.rbd3.in‚0 disk.rbd2.in‚0# proc.memory.rssMB‚ 2892.85546875* proc.memory.percent‚17.656588554382324 cpu‚25.656532827074116 network.recv‚ 1.613969e+06% disk./.usageGB‚1065.8846855163574 disk.rbd1.out‚2.515625 disk.drbd1777.in‚0 proc.cpu.threads‚17 disk.sda2.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda1.out‚0 disk.rbd0.out‚ 0.2978515625 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 1337.09375 disk.sda1.in‚0 disk.rbd0.in‚0) disk./.usagePercent‚71.53241558182279! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.in‚0.125! disk.rbd2.out‚399.20068359375 disk.sda.in‚0.125$ memory_percent‚92.11742877960205z‘Ú2³- _wandbruntime‚4654  _runtime‚4654}”u“É´Ã ' lr-AdamW/pg2‚0.00013711883534544902 trainer/global_step‚5049 % _timestamp‚1.7883938984089248e+09 _runtime‚4654.164910473 _step‚408 ' lr-AdamW/pg1‚0.00013711883534544902ªô¾ÎµÈ % _timestamp‚1.7883938984089248e+09 _runtime‚4654.164910473 _step‚408 ' lr-AdamW/pg1‚0.00013711883534544902 ' lr-AdamW/pg2‚0.00013711883534544902 trainer/global_step‚5049˜æ¬2¶- _wandbruntime‚4654  _runtime‚4654ÖÄÓì'·æO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚86594982 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚53063119 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 194298174 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚339927726875648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚339927743915008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚331676533684224 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚344090435244032 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚347033439206912 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚347033697847808 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚335816238434724 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883938984099786e+09 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚339927743915008 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4139495714600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚344090648462336 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚86594982 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚86594980 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4162904547328 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚409 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 107703194 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4162708368384 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 194320774 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚86594980 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚4654.166179505 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚342660098139648 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚53063116 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4162708368384 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚27128000 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚335816029398824 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 107725792 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 107725792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚496 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4373599708160 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4162904547328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 107703194 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 trainer/global_step‚5049 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4373365970944 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚409 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 107703194 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4162708368384 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 194320774 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚86594980 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚4654.166179505 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚342660098139648 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚53063116 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4162708368384 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚27128000 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚335816029398824 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 107725792 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 107725792 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚496 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4373599708160 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4162904547328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 107703194 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 trainer/global_step‚5049 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4373365970944 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚339927743915008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚331676533684224 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚344090435244032 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚347033439206912 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚347033697847808 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚335816238434724 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 % _timestamp‚1.7883938984099786e+09 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚339927743915008 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4139495714600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚344090648462336 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚86594982 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚86594980 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4162904547328 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚344156272671744 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚80206662 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚347100065553920 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4140481594296 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚86611502 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚339992782951936 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 194335566 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚335880631702456 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚5049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚86611500 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚344156485903360 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4163506759168 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4163702951424 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚53073150 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4140289597548 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 107724066 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚86611502 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚347099798537728 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 194335566 ZFDeviceStatsMonitor.on_train_batQþƒ/tch_end/allocated_bytes.large_pool.freed‚339992765912576 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 107746690 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚80206135 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27133509 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4163506759168 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 107724066 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚53073153 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27132985 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 194358192 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4374199805440 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 % _timestamp‚1.7883938991648676e+09 _runtime‚4654.921317397 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 _step‚410 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚342725623635968 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4163702951424 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4374441917952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507ùöÎû&»õM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4163702951424 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚53073150 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4140289597548 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 107724066 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚86611502 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚347099798537728 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 194335566 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚339992765912576 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 107746690 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚80206135 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27133509 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4163506759168 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 107724066 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚53073153 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27132985 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 194358192 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4374199805440 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 % _timestamp‚1.7883938991648676e+09 _runtime‚4654.921317397 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 _step‚410 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚342725623635968 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4163702951424 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4374441917952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚527 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 107746690 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚339992765912576 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚524 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚335880422666348 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚344156272671744 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚80206662 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚347100065553920 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4140481594296 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚86611502 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚339992782951936 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 194335566 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚335880631702456 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 trainer/global_step‚5049 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚86611500 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚344156485903360 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4163506759168 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616šŸ=Ša2¼- _wandbruntime‚4654  _runtime‚4654¶R^Z½Ô 2loss/FBt1/assignment_loss‚0.006368582683434296 1loss/SRqqt1/detection_loss‚1.7394423077681236 &loss/total_loss‚40.855461028392966 _runtime‚4654.922442628 2loss/SRqqt1/assignment_loss‚1.4639738600734356 0loss/FBt3/assignment_loss‚0.2153565481686118 /loss/FRt4/detection_loss‚2.5481546487127034 1loss/SRqqt4/detection_loss‚2.3380400930841763 -loss/FBt1/detection_loss‚3.06184850136439 % _timestamp‚1.7883938991666248e+09 1loss/SRqqt3/assignment_loss‚1.561387694317382 0loss/SRqqt2/detection_loss‚0.764128494516222 _step‚411 /loss/FRt3/assignment_loss‚2.719078213465018 0loss/FRt4/assignment_loss‚3.4965543108327046 .loss/FRt3/detection_loss‚2.723034389194895 1loss/SRqqt3/detection_loss‚1.8384855093061925 .loss/FBt4/detection_loss‚2.666608605782191 trainer/global_step‚5049 1loss/FBt2/assignment_loss‚0.08412994653379158 1loss/FBt4/assignment_loss‚0.47493542557931506 /loss/FRt2/detection_loss‚1.5116885248387213 2loss/SRqqt4/assignment_loss‚1.6771098647266627 .loss/FRt1/detection_loss‚2.180429415083399 /loss/FBt2/detection_loss‚0.4638608607237001 .loss/FBt3/detection_loss‚2.150708044117147 epoch‚8 0loss/FRt1/assignment_loss‚2.8941884280419816 /loss/FRt2/assignment_loss‚1.353726329779861 2loss/SRqqt2/assignment_loss‚0.9222224296989023{6j_¾Ù /loss/FRt3/assignment_loss‚2.719078213465018 0loss/FRt4/assignment_loss‚3.4965543108327046 .loss/FRt3/detection_loss‚2.723034389194895 1loss/SRqqt3/detection_loss‚1.8384855093061925 .loss/FBt4/detection_loss‚2.666608605782191 trainer/global_step‚5049 1loss/FBt2/assignment_loss‚0.08412994653379158 1loss/FBt4/assignment_loss‚0.47493542557931506 /loss/FRt2/detection_loss‚1.5116885248387213 2loss/SRqqt4/assignment_loss‚1.6771098647266627 .loss/FRt1/detection_loss‚2.180429415083399 /loss/FBt2/detection_loss‚0.4638608607237001 .loss/FBt3/detection_loss‚2.150708044117147 epoch‚8 0loss/FRt1/assignment_loss‚2.8941884280419816 /loss/FRt2/assignment_loss‚1.353726329779861 2loss/SRqqt2/assignment_loss‚0.9222224296989023 2loss/FBt1/assignment_loss‚0.006368582683434296 1loss/SRqqt1/detection_loss‚1.7394423077681236 &loss/total_loss‚40.855461028392966 _runtime‚4654.922442628 2loss/SRqqt1/assignment_loss‚1.4639738600734356 0loss/FBt3/assignment_loss‚0.2153565481686118 /loss/FRt4/detection_loss‚2.5481546487127034 1loss/SRqqt4/detection_loss‚2.3380400930841763 -loss/FBt1/detection_loss‚3.06184850136439 % _timestamp‚1.7883938991666248e+09 1loss/SRqqt3/assignment_loss‚1.561387694317382 0loss/SRqqt2/detection_loss‚0.764128494516222 _step‚411›fnêWœ¿‚(:‘ µóâÔðœ¸u gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.08' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.216000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0iP6EÀ‚(:º µóâÔ«ÏÍw disk.nvme0n1.in‚ 265.6328125 disk.sda2.out‚0 disk.dm-0.in‚ 265.6328125# disk.dm-0.out‚48962.97021484375 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.in‚0.125 disk.rbd1.out‚2.515625$ memory_percent‚92.12064743041992 network.sent‚ 1.0571569e+07 disk.sda1.in‚0 proc.cpu.threads‚17 disk.sda1.out‚0 disk.dm-1.in‚0.125 network.recv‚ 1.617597e+06& disk.nvme0n1.out‚48962.97021484375 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0% disk./.usageGB‚1065.8847045898438 disk.sda.out‚ 1338.4375 disk.drbd1777.in‚0 disk.rbd3.in‚0 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 1338.4375' proc.memory.availableMB‚ 1290.953125 cpu‚25.656370533787957 disk.sda2.in‚0 disk.rbd2.in‚0! disk.rbd2.out‚399.34130859375# proc.memory.rssMB‚ 2892.85546875* proc.memory.percent‚17.656588554382324 disk.sda.in‚0.125( disk.nvme0c0n1.out‚48962.97021484375 disk.rbd1.in‚0 disk.dm-1.out‚ 1338.4375) disk./.usagePercent‚71.532416861860458Zn’Á‚(:‡ ÄóâÔëý¾u gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚110.694' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.1388 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜íw»P‚(:Å ÄóâÔíš”y# disk.dm-0.out‚49061.25927734375 cpu‚25.656437803317225 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd2.in‚0& disk.nvme0n1.out‚49061.25927734375 disk.sda1.out‚0 disk.sda.out‚ 1352.8671875 disk.drbd1777.in‚0 disk.sda3.out‚ 1352.8671875 disk.dm-1.out‚ 1352.8671875% disk./.usageGB‚1065.8847274780273 disk.rbd0.in‚0( disk.nvme0c0n1.out‚49061.25927734375! disk.rbd2.out‚400.88818359375 disk.sda1.in‚0 network.sent‚ 1.0658118e+07 disk.rbd3.out‚ 2.349609375 proc.cpu.threads‚17 disk.sda.in‚0.125 network.recv‚ 1.623297e+06 disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125 disk.rbd1.in‚0 disk.rbd1.out‚2.515625# proc.memory.rssMB‚ 2892.85546875) disk./.usagePercent‚71.53241839790567 disk.rbd3.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.drbd1777.out‚0 disk.dm-1.in‚0.125$ memory_percent‚92.11757183074951) proc.memory.availableMB‚ 1291.45703125* proc.memory.percent‚17.656588554382324! disk.nvme0c0n1.in‚ 265.6328125!Ž“š‘Â(:† ÓóâÔ‘¦u gpu.0.gpu‚55.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚90.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.0248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ñ ÞlFÄ‚(:» ÓóâÔÂöÞw! disk.rbd2.out‚401.02880859375) disk./.usagePercent‚71.53230652261321% disk./.usageGB‚1065.8830604553223 disk.sda2.in‚0 disk.sda3.out‚ 1366.32421875 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0 disk.sda2.out‚0# disk.nvme0n1.out‚49159.76953125$ memory_percent‚92.99335479736328* proc.memory.percent‚17.656588554382324 cpu‚25.65648549431095 network.recv‚ 1.627829e+06 disk.sda1.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚0.125 disk.rbd1.out‚2.515625 disk.rbd2.in‚0 disk.sda.out‚ 1366.32421875 proc.cpu.threads‚17 disk.rbd0.out‚ 0.2978515625 disk.dm-0.out‚49159.76953125 disk.sda.in‚0.125 disk.dm-1.out‚ 1366.32421875% disk.nvme0c0n1.out‚49159.76953125 disk.sda3.in‚0.125& proc.memory.availableMB‚ 1147.96875 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd3.out‚ 2.349609375 disk.dm-0.in‚ 265.6328125# proc.memory.rssMB‚ 2892.85546875 network.sent‚ 1.0664574e+07 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd1.in‚03–2Å- _wandbruntime‚4696  _runtime‚4696›!ŒšÉÆà _step‚412 ' lr-AdamW/pg1‚0.00012396607223714328 ' lr-AdamW/pg2‚0.00012396607223714328 trainer/global_step‚5099 % _timestamp‚1.7883939407828195e+09 _runtime‚4696.539972448 -y¡ÎÇÈ % _timestamp‚1.7883939407828195e+09 _runtime‚4696.539972448 _step‚412 ' lr-AdamW/pg1‚0.00012396607223714328 ' lr-AdamW/pg2‚0.00012396607223714328 trainer/global_step‚5099œ€ÇÚó2È- _wandbruntime‚4696  _runtime‚4696ã2óì'ÉæO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚87420982 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 trainer/global_step‚5099 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _runtime‚4696.540069871 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚334856519952384 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚347382334457856 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚347382547676160 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 % _timestamp‚1.7883939407836008e+09 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚53564819 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚343179678722048 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚87420980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚347382334457856 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚350351725257728 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 196171126 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚53564816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4415377205760 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4202851914752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚343179695761408 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚80952200 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4415619331584 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _runtime‚4696.540069871 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚334856519952384 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚347382334457856 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚347382547676160 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 % _timestamp‚1.7883939407836008e+09 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚53564819 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚343179678722048 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚87420980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚347382334457856 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚350351725257728 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 196171126 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚53564816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4415377205760 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4202851914752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚343179695761408 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚80952200 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4415619331584 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚347448171885568 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚350418094353408 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 trainer/global_step‚5099 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚80967871 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 108771018 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚477 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 196231144 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚53574850 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚343244717758976 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚334920119336960 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚414 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚334920102297600 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚343244734798336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚350418357175296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4203454126592 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚343244717758976 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚347448385117184 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 108793642 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4416458723328 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚339100113214556 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4416220805120 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚80967391 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚53574853 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚339100322250664 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 _runtime‚4697.311835784 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4203650318848 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27393018 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788393941555255e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚87437502 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚343244734798336 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 196231144 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚343244717758976 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚334920119336960 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚414 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚334920102297600 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚343244734798336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚350418357175296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4203454126592 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚343244717758976 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚347448385117184 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 108793642 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4416458723328 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚339100113214556 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4416220805120 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚80967391 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚53574853 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚339100322250664 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 _runtime‚4697.311835784 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4203650318848 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27393018 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 $ _timestamp‚1.788393941555255e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚87437502 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚343244734798336 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 196231144 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚347448171885568 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚350418094353408 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 trainer/global_step‚5099 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚80967871 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 108771018 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚477 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 196231144 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚53574850 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0ž¸"ˆt2Î- _wandbruntime‚4697  _runtime‚4697Í2‚hZÏÔ /loss/FBt2/detection_loss‚0.5361923424665023 2loss/SRqqt4/assignment_loss‚1.6366277121835284 1loss/FBt1/assignment_loss‚0.45446776589637417 /loss/FRt2/assignment_loss‚1.241200523620302 1loss/SRqqt4/detection_loss‚2.3747670868086437 1loss/FBt2/assignment_loss‚0.15589753498880055 epoch‚8 % _timestamp‚1.7883939415571654e+09 _step‚415 /loss/FRt3/assignment_loss‚3.018528598953377 2loss/SRqqt1/assignment_loss‚1.4188904912728402 /loss/FRt1/detection_loss‚2.4972972363233565 /loss/FBt1/detection_loss‚2.7765144771999783 /loss/FBt4/detection_loss‚2.6753551363945007 2loss/SRqqt2/assignment_loss‚1.0130499066567669 /loss/FRt4/detection_loss‚2.7161788704895202 /loss/FBt3/detection_loss‚2.5973768457770348 %loss/total_loss‚43.17147328338248 /loss/FRt1/assignment_loss‚3.198446781495038 0loss/FBt3/assignment_loss‚0.1881042335045322 1loss/FBt4/assignment_loss‚0.46826827684223343 1loss/SRqqt2/detection_loss‚0.8528027199208736 trainer/global_step‚5099 _runtime‚ 4697.31296912 /loss/FRt3/detection_loss‚2.4125476730140774 1loss/SRqqt1/detection_loss‚2.0851115227987367 /loss/FRt4/assignment_loss‚3.653639578050183 2loss/SRqqt3/assignment_loss‚1.4050473248751103 .loss/FRt2/detection_loss‚1.689646037507125 /loss/SRqqt3/detection_loss‚2.10551460634304Ô""_ÐÙ 1loss/SRqqt4/detection_loss‚2.3747670868086437 1loss/FBt2/assignment_loss‚0.15589753498880055 epoch‚8 % _timestamp‚1.7883939415571654e+09 _step‚415 /loss/FRt3/assignment_loss‚3.018528598953377 2loss/SRqqt1/assignment_loss‚1.4188904912728402 /loss/FRt1/detection_loss‚2.4972972363233565 /loss/FBt1/detection_loss‚2.7765144771999783 /loss/FBt4/detection_loss‚2.6753551363945007 2loss/SRqqt2/assignment_loss‚1.0130499066567669 /loss/FRt4/detection_loss‚2.7161788704895202 /loss/FBt3/detection_loss‚2.5973768457770348 %loss/total_loss‚43.17147328338248 /loss/FRt1/assignment_loss‚3.198446781495038 0loss/FBt3/assignment_loss‚0.1881042335045322 1loss/FBt4/assignment_loss‚0.46826827684223343 1loss/SRqqt2/detection_loss‚0.8528027199208736 trainer/global_step‚5099 _runtime‚ 4697.31296912 /loss/FRt3/detection_loss‚2.4125476730140774 1loss/SRqqt1/detection_loss‚2.0851115227987367 /loss/FRt4/assignment_loss‚3.653639578050183 2loss/SRqqt3/assignment_loss‚1.4050473248751103 .loss/FRt2/detection_loss‚1.689646037507125 /loss/SRqqt3/detection_loss‚2.10551460634304 /loss/FBt2/detection_loss‚0.5361923424665023 2loss/SRqqt4/assignment_loss‚1.6366277121835284 1loss/FBt1/assignment_loss‚0.45446776589637417 /loss/FRt2/assignment_loss‚1.241200523620302ŸΈ½’Ñ‚(:‡ âóâÔïï¶v gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚142.071' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.4142 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0z‹DY%Ò‚(:· âóâÔ„áw disk.drbdÓ¹R;1777.in‚0 disk.sda2.in‚0 disk.dm-1.out‚ 1378.7734375! disk.rbd2.out‚402.56396484375 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125 disk.dm-1.in‚0.125% proc.memory.availableMB‚ 1147.9375# proc.memory.rssMB‚ 2892.85546875% disk.nvme0c0n1.out‚49160.16015625 network.sent‚ 1.066991e+07% disk./.usageGB‚1065.8850135803223 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.rbd0.out‚ 0.2978515625* proc.memory.percent‚17.656588554382324 network.recv‚ 1.631849e+06 disk.sda.out‚ 1378.7734375 disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-0.out‚49160.16015625$ memory_percent‚92.99354553222656 cpu‚25.656361215526637 disk.rbd1.out‚2.515625! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.in‚0.125 disk.rbd3.out‚ 2.349609375 disk.sda3.out‚ 1378.7734375 disk.sda.in‚0.125 disk.drbd1777.out‚0) disk./.usagePercent‚71.53243759847074# disk.nvme0n1.out‚49160.16015625 disk.rbd0.in‚0 disk.sda2.out‚0 proc.cpu.threads‚17‹軜Ó‚(:‘ ñóâÔâ·Çu gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚95.885' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚19.177000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0X!¹ŸIÔ‚(:¾ ñóâÔØŒ¯x disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.out‚0! disk.rbd2.out‚404.16943359375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.rbd3.out‚ 17.763671875 disk.sda3.in‚0.125 disk.sda3.out‚ 1394.609375 disk.dm-1.out‚ 1394.609375 disk.rbd1.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda1.in‚0 disk.rbd3.in‚0 disk.dm-1.in‚0.125 proc.cpu.threads‚17( disk./.usagePercent‚71.5324411825762 disk.sda.in‚0.125* proc.memory.percent‚17.656588554382324 network.recv‚ 1.637029e+06 disk.rbd2.in‚0 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚49258.7939453125) proc.memory.availableMB‚ 1147.90234375 cpu‚25.656386505502397 network.sent‚ 1.0756569e+07" disk.dm-0.out‚49258.7939453125$ disk./.usageGB‚1065.885066986084 disk.rbd1.out‚2.515625 disk.sda2.in‚0 disk.dm-0.in‚ 265.6328125% disk.nvme0n1.out‚49258.7939453125 disk.sda.out‚ 1394.609375$ memory_percent‚92.99376010894775# proc.memory.rssMB‚ 2892.85546875¹l:Ô2Õ- _wandbruntime‚4739  _runtime‚4739.•qÈÖ _step‚416 ' lr-AdamW/pg1‚0.00011142725877114515 ' lr-AdamW/pg2‚0.00011142725877114515 trainer/global_step‚5149 % _timestamp‚1.7883939834354517e+09 _runtime‚ 4739.19143167HMÍ×Ç _runtime‚ 4739.19143167 _step‚416 ' lr-AdamW/pg1‚0.00011142725877114515 ' lr-AdamW/pg2‚0.00011142725877114515 trainer/global_step‚5149 % _timestamp‚1.7883939834354517e+09 ;˜vª2Ø- _wandbruntime‚4739  _runtime‚4739!Epë'ÙåO \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚346431630568448 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚54066516 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚27647505 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚88246980 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚81714021 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 198066678 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚353670382537728 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚81714549 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 198044078 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚88246982 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚346431630568448 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚353670109216768 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚350674233671680 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 109797098 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚346431647607808 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4242799282176 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚350674446889984 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚88246982 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 109819696 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4457486348800 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚27648030 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚528 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚350674233671680 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚88246980 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚88246980 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚81714021 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 198066678 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚353670382537728 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚81714549 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 198044078 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚88246982 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚346431630568448 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚353670109216768 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚350674233671680 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚525 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 109797098 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚346431647607808 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4242799282176 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 248417280 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚350674446889984 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚88246982 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 109819696 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4457486348800 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚27648030 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568¡Oïïà2Û- _wandbruntime‚4739  _runtime‚4739Ôî«#ÜðM L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚346496669605376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4458571110400 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4219924233112 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚418 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27652718 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚88263502 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚350740284331008 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚353736744378368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚342320012798872 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4219732236364 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 109840594 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4243401494016 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 198081470 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚342319803762764 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚338100088565760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚54076550 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4458333192192 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4243597686272 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚350740071099392 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 198104096 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚505 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚349278148364288 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚5149 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚88263502 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚346496686644736 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚88263500 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚54076553 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚338100071526400 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚346496669605376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.curren=·Kt‚22624 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚81729268 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2Ä¡kxû&ÝõM JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚81729268 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚346496669605376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4458571110400 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4219924233112 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _step‚418 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27652718 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚88263502 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚350740284331008 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚502 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚353736744378368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚342320012798872 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4219732236364 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 109840594 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4243401494016 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 198081470 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚342319803762764 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚338100088565760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚54076550 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4458333192192 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4243597686272 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚350740071099392 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 198104096 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚505 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚349278148364288 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 trainer/global_step‚5149 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚88263502 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚346496686644736 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚88263500 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚54076553 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚338100071526400 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚346496669605376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0¢ÓvD?2Þ- _wandbruntime‚4739  _runtime‚4739BˆÐ^ßØ 0loss/FBt2/detection_loss‚0.47250452186195074 % _timestamp‚1.7883939842136657e+09 0loss/FRt1/assignment_loss‚2.7746167883768185 1loss/FBt1/assignment_loss‚0.11883603852182109 1loss/SRqqt2/detection_loss‚1.0133298731480653 0loss/FRt4/assignment_loss‚3.8454666097285384 /loss/FRt2/detection_loss‚1.6327649152909334 /loss/FBt1/detection_loss‚2.2960556494562248 .loss/FBt3/detection_loss‚2.267977925993147 /loss/FBt4/detection_loss‚3.1255470109837398 _runtime‚4739.969505183 2loss/SRqqt4/assignment_loss‚1.4834095402842475 1loss/FBt2/assignment_loss‚0.15830490291339047 1loss/SRqqt1/detection_loss‚1.9359833834986937 1loss/SRqqt3/detection_loss‚2.1387344947721383 _step‚419 2loss/SRqqt1/assignment_loss‚1.2386150087965162 2loss/SRqqt3/assignment_loss‚1.4640386968473844 .loss/FRt1/detection_loss‚2.402928590283289 1loss/SRqqt4/detection_loss‚2.4206616817012665 epoch‚8 trainer/global_step‚5149 .loss/FRt3/detection_loss‚2.368523553465352 0loss/FRt2/assignment_loss‚1.3821763333219748 -loss/FRt4/detection_loss‚2.71543709100303 2loss/SRqqt2/assignment_loss‚0.8490226735360921 0loss/FBt4/assignment_loss‚0.4569169761588065 &loss/total_loss‚41.984141652328105 /loss/FRt3/assignment_loss‚3.221961743903883 1loss/FBt3/assignment_loss‚0.20032764848080356w¤1PcàÝ -loss/FRt4/detection_loss‚2.71543709100303 2loss/SRqqt2/assignment_loss‚0.8490226735360921 0loss/FBt4/assignment_loss‚0.4569169761588065 &loss/total_loss‚41.984141652328105 /loss/FRt3/assignment_loss‚3.221961743903883 1loss/FBt3/assignment_loss‚0.20032764848080356 0loss/FBt2/detection_loss‚0.47250452186195074 % _timestamp‚1.7883939842136657e+09 0loss/FRt1/assignment_loss‚2.7746167883768185 1loss/FBt1/assignment_loss‚0.11883603852182109 1loss/SRqqt2/detection_loss‚1.0133298731480653 0loss/FRt4/assignment_loss‚3.8454666097285384 /loss/FRt2/detection_loss‚1.6327649152909334 /loss/FBt1/detection_loss‚2.2960556494562248 .loss/FBt3/detection_loss‚2.267977925993147 /loss/FBt4/detection_loss‚3.1255470109837398 _runtime‚4739.969505183 2loss/SRqqt4/assignment_loss‚1.4834095402842475 1loss/FBt2/assignment_loss‚0.15830490291339047 1loss/SRqqt1/detection_loss‚1.9359833834986937 1loss/SRqqt3/detection_loss‚2.1387344947721383 _step‚419 2loss/SRqqt1/assignment_loss‚1.2386150087965162 2loss/SRqqt3/assignment_loss‚1.4640386968473844 .loss/FRt1/detection_loss‚2.402928590283289 1loss/SRqqt4/detection_loss‚2.4206616817012665 epoch‚8 trainer/global_step‚5149 .loss/FRt3/detection_loss‚2.368523553465352 0loss/FRt2/assignment_loss‚1.3821763333219748£L¯bÅá‚(:’ €ôâԾĿu gpu.0.gpu‚55.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚102.205' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚20.441000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ík‚Tâ‚(:É €ôâÔ囜x disk.sda.in‚0.125 disk.sda1.out‚0 disk.drbd1777.in‚0 network.sent‚ 1.0762717e+07# disk.dm-0.out‚49279.06787109375 disk.dm-1.in‚0.125 disk.rbd3.in‚0 disk.rbd3.out‚ 17.841796875 cpu‚25.656265137766464 network.recv‚ 1.641973e+06 disk.sda3.out‚ 1395.43359375! disk.nvme0c0n1.in‚ 265.6328125) proc.memory.availableMB‚ 1147.41015625 disk.rbd2.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda2.in‚0$ memory_percent‚92.99676418304443) disk./.usagePercent‚71.53663151389667 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚49279.06787109375 disk.sda2.out‚0 proc.cpu.threads‚17 disk.sda3.in‚0.125 disk.rbd1.in‚0 disk.rbd1.out‚2.515625! disk.rbd2.out‚404.19677734375 disk.nvme0n1.in‚ 265.6328125 disk.sda.out‚ 1395.43359375 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625# proc.memory.rssMB‚ 2892.85546875* proc.memory.percent‚17.656588554382324% disk./.usageGB‚1065.9475059509277 disk.dm-1.out‚ 1395.43359375& disk.nvme0n1.out‚49279.06787109375aG§‘ã‚(:† ôâԼļv gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚97.239' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.4478 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0RÑŽNä‚(:à ôâÔÄ·w& disk.nvme0c0n1.out‚49459.005859375 network.recv‚ 1.648184e+06) disk./.usagePercent‚71.53244067056114! disk.rbd2.out‚405.84912109375 disk.nvme0n1.in‚ 265.6328125 disk.dm-0.in‚ 265.6328125 disk.dm-1.out‚ 1411.43359375 disk.rbd1.out‚2.515625# proc.memory.rssMB‚ 2892.85546875 network.sent‚ 1.0850637e+07 disk.drbd1777.in‚0 disk.rbd3.out‚ 79.076171875 disk.dm-1.in‚0.125$ memory_percent‚92.99404621124268) proc.memory.availableMB‚ 1147.85546875 proc.cpu.threads‚17$ disk.nvme0n1.out‚49459.005859375 disk.sda1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.rbd3.in‚0! disk.dm-0.out‚49459.005859375 disk.sda3.in‚0.125 disk.sda.in‚0.125 disk.rbd2.in‚0 disk.sda1.in‚0 disk.drbd1777.out‚0 disk.sda2.out‚0% disk./.usageGB‚1065.8850593566895 disk.sda3.out‚ 1411.43359375 disk.rbd1.in‚0* proc.memory.percent‚17.656588554382324 cpu‚25.656254489879462 disk.sda.out‚ 1411.43359375! disk.nvme0c0n1.in‚ 265.6328125n€ßå‚(:… žôâÔëÕËu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚91.295' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.259 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0×8é5Sæ‚(:È žôâÔ½ƒŒy disk.rbd3.out‚79.50830078125 disk.nvme0n1.in‚ 265.6328125 disk.drbd1777.out‚0 disk.sda2.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda.out‚ 1427.33984375' proc.memory.availableMB‚ 1147.859375* proc.memory.percent‚17.656588554382324 network.sent‚ 1.0855981e+07 disk.sda3.in‚0.125 disk.sda.in‚0.125 disk.rbd0.out‚ 0.2978515625$ memory_percent‚92.99402236938477 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.drbd1777.in‚0 cpu‚25.65626836294027) disk./.usagePercent‚71.53244527869677 disk.sda3.out‚ 1427.33984375 disk.dm-1.in‚0.125# proc.memory.rssMB‚ 2892.85546875 disk.rbd0.in‚0! disk.nvme0c0n1.in‚ 265.6328125( disk.nvme0c0n1.out‚49557.89208984375 disk.dm-1.out‚ 1427.33984375! disk.rbd2.out‚405.87255859375& disk.nvme0n1.out‚49557.89208984375 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.out‚0 proc.cpu.threads‚17 network.recv‚ 1.651795e+06% disk./.usageGB‚1065.8851280212402# disk.dm-0.out‚49557.89208984375 disk.rbd1.out‚2.515625ÆZRM2ç- _wandbruntime‚4781  _runtime‚4781—GÀEÇèÁ % _timestamp‚1.7883940260943198e+09 _runtime‚4781.850312979 _step‚420 & lr-AdamW/pg1‚9.951286399278067e-05 & lr-AdamW/pg2‚9.951286399278067e-05 trainer/global_step‚5199MûáÌéÆ & lr-AdamW/pg1‚9.951286399278067e-05 & lr-AdamW/pg2‚9.951286399278067e-05 trainer/global_step‚5199 % _timestamp‚1.7883940260943198e+09 _runtime‚4781.850312979 _step‚420¤D®32ê- _wandbruntime‚4781  _runtime‚4781)M ëì'ëæO UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚353966132885504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4282550470656 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 % _timestamp‚1.7883940260954227e+09 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚349683599454208 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4499866435584 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4258659672824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚349683599454208 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚356988530382336 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 110844050 _runtime‚4781.851604153 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚341216441370624 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚27907689 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚89072980 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚82475396 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 110844050 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚352488922587648 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚353966346103808 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4282746649600 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 199939630 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚82475908 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚353966132885504 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚27907180 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚345475310079348 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4258851669364 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚89072982 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚356988789023232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4282746649600 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 199917030 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚349683582414848 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚54568216 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚89072982 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5199 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 110866648 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4499632698368 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚341216458409984 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4258659672824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚349683599454208 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚356988530382336 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 110844050 _runtime‚4781.851604153 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚341216441370624 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚27907689 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚89072980 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚82475396 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 110844050 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚352488922587648 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚353966346103808 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4282746649600 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 199939630 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚82475908 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚353966132885504 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚27907180 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocatµmW7ed‚345475310079348 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4258851669364 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚89072982 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚356988789023232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4282746649600 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 199917030 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚349683582414848 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚512 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚54568216 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚89072982 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5199 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 110866648 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4499632698368 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚341216458409984 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 JDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚89089500 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚349748638491136 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚341280040755200 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚357054901839360 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚5199 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4283545053696 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚89089500 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚497 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚349748621451776 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 110887546 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚54578253 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4259453555772 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚341280057794560 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 110864922 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚349748638491136 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚82490598 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 110864922 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 199954422 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4283348861440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27912842 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27912348 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 199977048 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507MNQ¥û&ïõM QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4500712382976 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚349748638491136 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚82490598 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 110864922 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 199954422 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4283348861440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚27912842 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚27912348 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 199977048 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4500478659072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 % _timestamp‚1.7883940268818593e+09 _step‚422 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚82491095 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚494 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚89089500 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚349748638491136 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚341280040755200 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚357054901839360 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚5199 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4283545053696 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚89089500 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚497 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚349748621451776 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 110887546 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚54578253 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4259453555772 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚341280057794560 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 110864922 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚83237231 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚348694791591656 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚357258245317632 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4541967681024 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚357258032099328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 111913600 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚344396410599424 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 _runtime‚ 4824.34866708 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 111891002 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚352935534261248 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚357258032099328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 201812582 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚89898980 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚344396427638784 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 111891002 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚89898982 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚352935551300608 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 % _timestamp‚1.7883940685924673e+09 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚83236694 trainer/global_step‚5249 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚534 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _step‚425 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4322694017024 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚89898980 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4322694017024 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28166778 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4322497838080 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048y×ÿÚð'þêO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28166778 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4322497838080 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 260738048 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚357258245317632 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚89898982 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚348695000627556 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚360307165084672 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4541731846656 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚83237231 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚348694791591656 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚357258245317632 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4541967681024 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚357258032099328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 111913600 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚344396410599424 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 _runtime‚ 4824.34866708 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 111891002 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚352935534261248 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚357258032099328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 201812582 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚89898980 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚344396427638784 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 111891002 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚89898982 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚352935551300608 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 % _timestamp‚1.7883940685924673e+09 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚83236694 trainer/global_step‚5249 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚534 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 _step‚425 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4322694017024 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚89898980 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4322694017024 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360©{•Ñ2ÿ- _wandbruntime‚4825  _runtime‚4825sïgö&€ðM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 111911874 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚55079953 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚344460009984000 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚55079950 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4323296228864 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚353000573298176 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 201850000 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 201827374 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚353000573298176 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4323492421120 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4542819979264 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4323296228864 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚355830722899968 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 111934498 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚83251918 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚348759184859180 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚344460027023360 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4299174875180 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 201827374 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚83252443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚355830697996288 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚353000590337536 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 201850000 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚357324082758656 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚357323869527040 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4299366871928 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4323492421120 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚357324082758656 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚348759393895288 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28171968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚89915502 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚89915502 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4542579963904 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚89915500–@åŒZõM VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 111911874 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚89915502 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚89915502 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4542579963904 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚89915500 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 111911874 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚55079953 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚344460009984000 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚55079950 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4323296228864 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 264919040 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚353000573298176 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 201850000 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 201827374 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚353000573298176 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4323492421120 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4542819979264 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4323296228864 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚355830722899968 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 111934498 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚83251918 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚348759184859180 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚344460027023360 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4299174875180 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 201827374 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚83252443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚355830697996288 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚353000590337536 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 201850000 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚357324082758656 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚357323869527040 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4299366871928 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 240015360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4323492421120 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚357324082758656 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚348759393895288 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28171968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _runtime‚4867.013993457 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 203662934 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 112937954 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚360549931313152 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚55571616 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚363625446737408 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚356187503147008 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚351914482139864 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4338102311640 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚90724982 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4362641384448 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚360550144531456 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4338294308180 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚90724980 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4362641384448 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4362445205504 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚83997811 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4584243644416 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 trainer/global_step‚5299 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 112937954 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚83998344 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚351914691175764 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 112960552 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚359041472219648 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824ܧkêO [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4584243644416 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 trainer/global_step‚5299 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 112937954 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚83998344 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚351914691175764 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 112960552 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚359041472219648 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚347576379828224 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _step‚429 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚530 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 203685534 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚359041447315968 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4583999421440 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚363625715864064 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚360549931313152 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚356187486107648 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _runtime‚4867.013993457 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 203662934 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 112937954 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚360549931313152 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚55571616 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚363625446737408 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚356187503147008 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚351914482139864 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4338102311640 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚90724982 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4362641384448 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚360550144531456 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4338294308180 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚90724980 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4362641384448 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4362445205504 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚83997811­ Æo2‘- _wandbruntime‚4867  _runtime‚4867x¢qíõ&’ïM 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚363692080641536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚5299 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4585082925568 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚351978875407388 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚90741502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚359106972812288 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚84012867 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚55581653 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚90741502 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 112958826 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4363243596288 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 112981450 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 $ _timestamp‚1.788394112013301e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚356252525144576 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 203722952 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 203700326 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4363243596288 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚90741500 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _step‚430 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚360615981972480 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚356252525144576 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 203722952 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28431217 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _runtime‚4867.770097351 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚360615981972480 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚356252542183936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚363691813625344 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚28431733 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4339088191336 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 112958826 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚55581650 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚356252542183936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚360615768740864 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 203700326—ýj‹ú&“ôM MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚363691813625344 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚28431733 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4339088191336 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 112958826 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚55581650 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚356252542183936 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚360615768740864 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 203700326 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚363692080641536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 trainer/global_step‚5299 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4585082925568 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚351978875407388 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚90741502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚359106972812288 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚84012867 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚55581653 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚90741502 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 112958826 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4363243596288 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 112981450 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 $ _timestamp‚1.788394112013301e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚356252525144576 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 203722952 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 203700326 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4363243596288 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚90741500 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 _step‚430 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚360615981972480 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚356252525144576 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 203722952 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28431217 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 _runtime‚4867.770097351 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚360615981972480 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚356252542183936 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0®<žm°2”- _wandbruntime‚4867  _runtime‚4867Æ•’Ü\•Ö 0loss/FRt2/assignment_loss‚1.5393600904189788 2loss/SRqqt1/assignment_loss‚1.4907744998888424 1loss/SRqqt2/detection_loss‚1.1755348511816313 trainer/global_step‚5299 /loss/FRt2/detection_loss‚1.7295686162269868 .loss/FBt4/detection_loss‚2.900758770379153 /loss/FRt3/detection_loss‚2.2984789658499802 epoch‚8 2loss/SRqqt3/assignment_loss‚1.4627119655813183 2loss/SRqqt4/assignment_loss‚2.1360921920151323 _step‚431 % _timestamp‚1.7883941120160165e+09 2loss/SRqqt2/assignment_loss‚0.9602077339950483 .loss/FRt1/detection_loss‚2.327370977272158 1loss/SRqqt3/detection_loss‚1.5933064147494398 /loss/FBt1/detection_loss‚2.6473236203193666 /loss/FBt2/detection_loss‚0.7126202436180652 .loss/FBt3/detection_loss‚2.266250772608651 1loss/FBt2/assignment_loss‚0.11476121179387394 1loss/FBt3/assignment_loss‚0.17745555906892857 0loss/SRqqt4/detection_loss‚2.298555147003483 &loss/total_loss‚41.823670741985765 0loss/FRt3/assignment_loss‚2.9253486977971117 /loss/FRt4/assignment_loss‚3.592799024148421 1loss/FBt1/assignment_loss‚0.22257069628103637 1loss/FBt4/assignment_loss‚0.07204677216662442 /loss/FRt4/detection_loss‚2.6031435660340567 .loss/FRt1/assignment_loss‚3.06431073922178 0loss/SRqqt1/detection_loss‚1.512319614365697 _runtime‚4867.771877776" Ó&a–Û 2loss/SRqqt4/assignment_loss‚2.1360921920151323 _step‚431 % _timestamp‚1.7883941120160165e+09 2loss/SRqqt2/assignment_loss‚0.9602077339950483 .loss/FRt1/detection_loss‚2.327370977272158 1loss/SRqqt3/detection_loss‚1.5933064147494398 /loss/FBt1/detection_loss‚2.6473236203193666 /loss/FBt2/detection_loss‚0.7126202436180652 .loss/FBt3/detection_loss‚2.266250772608651 1loss/FBt2/assignment_loss‚0.11476121179387394 1loss/FBt3/assignment_loss‚0.17745555906892857 0loss/SRqqt4/detection_loss‚2.298555147003483 &loss/total_loss‚41.823670741985765 0loss/FRt3/assignment_loss‚2.9253486977971117 /loss/FRt4/assignment_loss‚3.592799024148421 1loss/FBt1/assignment_loss‚0.22257069628103637 1loss/FBt4/assignment_loss‚0.07204677216662442 /loss/FRt4/detection_loss‚2.6031435660340567 .loss/FRt1/assignment_loss‚3.06431073922178 0loss/SRqqt1/detection_loss‚1.512319614365697 _runtime‚4867.771877776 0loss/FRt2/assignment_loss‚1.5393600904189788 2loss/SRqqt1/assignment_loss‚1.4907744998888424 1loss/SRqqt2/detection_loss‚1.1755348511816313 trainer/global_step‚5299 /loss/FRt2/detection_loss‚1.7295686162269868 .loss/FBt4/detection_loss‚2.900758770379153 /loss/FRt3/detection_loss‚2.2984789658499802 epoch‚8 2loss/SRqqt3/assignment_loss‚1.4627119655813183¯cVõ4—‚(:’ ‡õâÔèœÑu gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚136.415' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.282999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,²÷+D˜‚(:¹ ‡õâԕ܃x# proc.memory.rssMB‚ 2892.85546875* proc.memory.percent‚17.656588554382324) disk./.usagePercent‚71.53436379915836 disk.dm-1.in‚0.125 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0 disk.rbd0.in‚0 disk.sda.in‚0.125 network.recv‚ 1.684709e+06 disk.rbd3.out‚84.05908203125 disk.rbd1.in‚0 disk.rbd1.out‚2.515625! disk.nvme0n1.out‚ 50061.484375 proc.cpu.threads‚17 disk.sda2.in‚0 disk.dm-0.in‚ 265.6328125 disk.dm-0.out‚ 50061.484375 disk.sda2.out‚0 disk.rbd3.in‚0 disk.dm-1.out‚ 1500.01171875 disk.rbd0.out‚ 0.2978515625 network.sent‚ 1.1060123e+07# disk.nvme0c0n1.out‚ 50061.484375 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.sda.out‚ 1500.01171875 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd2.in‚0 disk.rbd2.out‚ 412.572265625 cpu‚25.65599688225319% disk./.usageGB‚1065.9137153625488$ memory_percent‚92.99108982086182) proc.memory.availableMB‚ 1148.33984375 disk.sda3.in‚0.125 disk.sda3.out‚ 1500.01171875ĥ咙‚(:‡ –õâÔÓÏÅu gpu.0.gpu‚20.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚153.088' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.6176 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’FAš‚(:¶ –õâÔµœ—y disk.rbd0.in‚0 disk.rbd1.in‚0 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd3.out‚84.35400390625 network.recv‚ 1.690005e+06 disk.rbd3.in‚0 disk.sda3.in‚0.125 disk.dm-1.in‚0.125 disk.rbd1.out‚2.515625$ disk./.usageGB‚1065.886074066162 disk.sda.out‚ 1513.765625 disk.rbd2.out‚ 414.111328125# proc.memory.rssMB‚ 2892.85546875 proc.cpu.threads‚17) disk./.usagePercent‚71.53250876856526 disk.dm-0.in‚ 265.6328125 disk.dm-1.out‚ 1513.765625' proc.memory.availableMB‚ 1148.578125 disk.sda3.out‚ 1513.765625 disk.sda.in‚0.125% disk.nvme0c0n1.out‚50162.05859375 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-0.out‚50162.05859375 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0* proc.memory.percent‚17.656588554382324 cpu‚25.656016719407393 network.sent‚ 1.1146761e+07 disk.rbd2.in‚0# disk.nvme0n1.out‚50162.05859375# memory_percent‚92.9896354675293 disk.sda1.in‚0ü)0›‚(:’ ¥õâÔ¸ÞËu gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚147.795' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.558999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0-wmKœ‚(:À ¥õâÔ»·¾x disk.sda3.out‚ 1531.421875 disk.rbd2.in‚0" memory_percent‚92.991042137146* proc.memory.percent‚17.656588554382324 network.sent‚ 1.1152124e+07 network.recv‚ 1.693611e+06$ disk./.usageGB‚1065.886074066162 disk.sda2.in‚0 disk.rbd3.out‚84.35400390625# disk.dm-0.out‚50162.96630859375 disk.dm-1.in‚0.125 disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd2.out‚ 414.255859375 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0) proc.memory.availableMB‚ 1148.34765625 cpu‚25.655906236488185 disk.rbd3.in‚0 proc.cpu.threads‚17( disk.nvme0c0n1.out‚50162.96630859375 disk.rbd1.in‚0 disk.sda1.out‚0 disk.sda3.in‚0.125 disk.dm-1.out‚ 1531.421875 disk.rbd1.out‚2.515625 disk.sda.out‚ 1531.421875) disk./.usagePercent‚71.53250876856526 disk.dm-0.in‚ 265.6328125 disk.nvme0n1.in‚ 265.6328125 disk.sda.in‚0.125 disk.rbd0.in‚0# proc.memory.rssMB‚ 2892.85546875 disk.sda2.out‚0& disk.nvme0n1.out‚50162.96630859375æÖ2- _wandbruntime‚4909  _runtime‚4909Vj^Åž¿ % lr-AdamW/pg2‚6.76130126463824e-05 trainer/global_step‚5349 % _timestamp‚1.7883941535461624e+09 _runtime‚4909.302134762 _step‚432 % lr-AdamW/pg1‚6.76130126463824e-05™’=}~ŸÄ _runtime‚4909.302134762 _step‚432 % lr-AdamW/pg1‚6.76130126463824e-05 % lr-AdamW/pg2‚6.76130126463824e-ñpL05 trainer/global_step‚5349 % _timestamp‚1.7883941535461624e+09°¢€âÚ2 - _wandbruntime‚4909  _runtime‚4909­Xê'¡äO H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚362317747035648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚355134381723972 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚350756366096384 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5349 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚91550982 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4402392572928 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚84758436 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788394153547295e+09 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 113984906 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 114007504 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚558 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚362317722131968 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚363842043745280 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚366943970320896 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚91550982 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚56073319 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 114007504 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚359439437954048 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚363841830526976 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4402588751872 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚56073316 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4625972770816 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚359439454993408 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4402392572928 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚363841830526976 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚359439437954048 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 113984906 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4402588751872 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4626223285248 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 205558486 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚84758997 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28685120 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750™‚=2ï'¢éO \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚362317747035648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚355134381723972 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚350756366096384 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5349 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚91550982 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4402392572928 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚84758436 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 $ _timestamp‚1.788394153547295e+09 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 113984906 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 114007504 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚558 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 275418112 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚362317722131968 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚363842043745280 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚366943970320896 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚91550982 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚56073319 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 114007504 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚359439437954048 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚363841830526976 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4402588751872 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚56073316 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4625972770816 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚359439454993408 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4402392572928 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚363841830526976 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚359439437954048 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 113984906 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4402588751872 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4626223285248 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 205558486 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚84758997 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28685120 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0±YÚ—2£- _wandbruntime‚4910  _runtime‚4910NòA¿ô&¤îM REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚91567502 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4403190963712 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 205573278 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4626820630016 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4627064839680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4403387155968 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4403190963712 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 205595904 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 205595904 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚91567500 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚84773563 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 # _timestamp‚1.78839415431277e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚363907881186304 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 114005778 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28690213 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 114005778 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚84774087 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚359504476990976 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚91567500 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚91567502 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚363907667954688 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 trainer/global_step‚5349 _step‚434 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚4910.069379096 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 114028402 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚367010068258304 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750AX'R¥óM L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚84773563 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 # _timestamp‚1.78839415431277e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚363907881186304 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 114005778 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚28690213 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 114005778 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚84774087 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚359504476990976 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚91567500 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚91567502 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚363907667954688 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 trainer/global_step‚5349 _step‚434 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚521 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 _runtime‚4910.069379096 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 114028402 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚367010068258304 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚91567502 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4403190963712 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 205573278 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4626820630016 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4627064839680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4403387155968 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4403190963712 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 205595904 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 205595904 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚91567500 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0²ʼnqH2¦- _wandbruntime‚4910  _runtime‚4910XQç`§Ú trainer/global_step‚5349 _runtime‚4910.070663974 2loss/SRqqt2/assignment_loss‚1.1262234756314353 2loss/SRqqt3/assignment_loss‚1.6149129778654738 1loss/SRqqt4/assignment_loss‚1.895824509789236 /loss/FRt1/detection_loss‚2.3210220994408597 2loss/SRqqt1/assignment_loss‚1.3662524326592338 1loss/FBt1/assignment_loss‚0.15853480467236256 /loss/FBt4/detection_loss‚2.6793217225508257 0loss/FRt3/assignment_loss‚2.8604744731540412 1loss/FBt3/assignment_loss‚0.42737033157795346 .loss/FRt4/detection_loss‚2.397690228827588 .loss/FBt1/detection_loss‚2.073749817469541 0loss/FRt1/assignment_loss‚3.0258556413896307 0loss/FRt2/assignment_loss‚1.8345709970841806 1loss/SRqqt3/detection_loss‚1.9550476731224493 1loss/SRqqt2/detection_loss‚0.7636116630272239 1loss/SRqqt4/detection_loss‚2.4565834091044962 % _timestamp‚1.7883941543148448e+09 0loss/FRt4/assignment_loss‚3.5838294741395234 /loss/FBt3/detection_loss‚2.5160910420947604 &loss/total_loss‚42.144817671621574 _step‚435 1loss/FBt2/assignment_loss‚0.13693666647786937 /loss/FRt2/detection_loss‚1.8187154916813597 /loss/FRt3/detection_loss‚2.5870539462062676 1loss/SRqqt1/detection_loss‚1.7134724747803476 /loss/FBt2/detection_loss‚0.6331186034032051 epoch‚8 0loss/FBt4/assignment_loss‚0.1985537154717101-Œëe¨ß trainer/global_step‚5349 _runtime‚4910.070663974 2loss/SRqqt2/assignment_loss‚1.1262234756314353 2loss/SRqqt3/assignment_loss‚1.6149129778654738 1loss/SRqqt4/assignment_loss‚1.895824509789236 /loss/FRt1/detection_loss‚2.3210220994408597 2loss/SRqqt1/assignment_loss‚1.3662524326592338 1loss/FBt1/assignment_loss‚0.15853480467236256 /loss/FBt4/detection_loss‚2.6793217225508257 0loss/FRt3/assignment_loss‚2.8604744731540412 1loss/FBt3/assignment_loss‚0.42737033157795346 .loss/FRt4/detection_loss‚2.397690228827588 .loss/FBt1/detection_loss‚2.073749817469541 0loss/FRt1/assignment_loss‚3.0258556413896307 0loss/FRt2/assignment_loss‚1.8345709970841806 1loss/SRqqt3/detection_loss‚1.9550476731224493 1loss/SRqqt2/detection_loss‚0.7636116630272239 1loss/SRqqt4/detection_loss‚2.4565834091044962 % _timestamp‚1.7883941543148448e+09 0loss/FRt4/assignment_loss‚3.5838294741395234 /loss/FBt3/detection_loss‚2.5160910420947604 &loss/total_loss‚42.144817671621574 _step‚435 1loss/FBt2/assignment_loss‚0.13693666647786937 /loss/FRt2/detection_loss‚1.8187154916813597 /loss/FRt3/detection_loss‚2.5870539462062676 1loss/SRqqt1/detection_loss‚1.7134724747803476 /loss/FBt2/detection_loss‚0.6331186034032051 epoch‚8 0loss/FBt4/assignment_loss‚0.1985537154717101³–’PC©‚(:’ ´õâԱؠu gpu.0.gpu‚51.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚134.684' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.936799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0—ž .Pª‚(:Å ´õâÔįÌw disk.dm-0.in‚ 265.6328125 disk.rbd1.out‚2.515625 disk.rbd2.in‚0 disk.drbd1777.in‚0$ memory_percent‚92.99297332763672& proc.memory.availableMB‚ 1148.03125 disk.sda2.out‚0 disk.sda3.in‚0.125 disk.sda3.out‚ 1531.93359375! disk.nvme0c0n1.in‚ 265.6328125( disk.nvme0c0n1.out‚50263.50341796875 cpu‚25.655898053270857 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd2.out‚ 415.791015625# proc.memory.rssMB‚ 2892.85546875 disk.rbd3.in‚0 network.sent‚ 1.1158268e+07) disk./.usagePercent‚71.53250876856526 disk.rbd3.out‚84.38525390625 disk.dm-1.out‚ 1531.93359375 disk.nvme0n1.in‚ 265.6328125 disk.rbd0.out‚ 0.2978515625* proc.memory.percent‚17.656588554382324& disk.nvme0n1.out‚50263.50341796875 disk.sda.in‚0.125 proc.cpu.threads‚17 disk.dm-1.in‚0.125 disk.sda1.in‚0 disk.rbd0.in‚0 network.recv‚ 1.698128e+06$ disk./.usageGB‚1065.886074066162# disk.dm-0.out‚50263.50341796875 disk.sda.out‚ 1531.93359375 disk.sda1.out‚0 disk.drbd1777.out‚0mÈ6ö’«‚(:‡ ÃõâÔŠ¹Íu gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.087' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.0174 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0׶ÛÓ?¬‚(:´ ÃõâÔ§«àx disk.rbd2.out‚ 417.505859375 disk.rbd1.out‚2.515625) proc.memory.availableMB‚ 1148.02734375 network.sent‚ 1.1246086e+07% disk./.usageGB‚1065.8862190246582 disk.sda3.out‚1544.25# proc.memory.rssMB‚ 2892.85546875 disk.sda.out‚1544.25 disk.rbd0.out‚ 0.2978515625 disk.dm-1.in‚0.125 cpu‚25.655917959435985 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.in‚0 disk.sda2.out‚0" disk.dm-0.out‚50362.0869140625 disk.rbd0.in‚0 disk.sda3.in‚0.125 disk.rbd1.in‚0 network.recv‚ 1.704293e+06 disk.nvme0n1.in‚ 265.6328125 disk.sda.in‚0.125 disk.sda1.in‚0 disk.drbd1777.out‚0' disk.nvme0c0n1.out‚50362.0869140625 disk.rbd3.in‚0) disk./.usagePercent‚71.53251849685157 disk.rbd2.in‚0% disk.nvme0n1.out‚50362.0869140625 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.rbd3.out‚84.42041015625 disk.dm-1.out‚1544.25 disk.dm-0.in‚ 265.6328125$ memory_percent‚92.99299716949463* proc.memory.percent‚17.656588554382324é‡Ð­‚(:’ ÒõâÔëÓ”u gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚137.639' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.527800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0€:!N®‚(:à ÒõâÔÍùÍx disk.sda1.out‚0& disk.nvme0c0n1.out‚50399.123046875 disk.rbd1.out‚2.515625# proc.memory.rssMB‚ 2892.85546875 disk.sda.out‚ 1544.71484375 disk.rbd3.in‚0 disk.sda3.in‚0.125 disk.rbd2.out‚ 417.533203125 disk.nvme0n1.in‚ 265.6328125 cpu‚25.655852539586085% disk./.usageGB‚1065.9398345947266! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.in‚0* proc.memory.percent‚17.656588554382324 disk.rbd0.out‚ 0.2978515625 disk.sda2.out‚0! disk.dm-0.out‚50399.123046875 disk.sda3.out‚ 1544.71484375 disk.dm-1.out‚ 1544.71484375 disk.rbd1.in‚0$ memory_percent‚92.99299716949463) proc.memory.availableMB‚ 1148.02734375 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.dm-1.in‚0.125 proc.cpu.threads‚17) disk./.usagePercent‚71.53611668274532 disk.rbd2.in‚0 network.recv‚ 1.708329e+06 disk.sda1.in‚0 disk.rbd3.out‚84.54931640625 disk.dm-0.in‚ 265.6328125 network.sent‚ 1.1251527e+07 disk.sda.in‚0.125$ disk.nvme0n1.out‚50399.123046875§;2¯- _wandbruntime‚4951  _runtime‚4951­ÊÛÆ°À & lr-AdamW/pg1‚5.829043421542946e-05 & lr-AdamW/pg2‚5.829043421542946e-05 trainer/global_step‚5399 $ _timestamp‚1.788394196238799e+09 _runtime‚4951.994857841 _step‚436 °N˱Š_runtime‚4951.994857841 _step‚436 & lr-AdamW/pg1‚5.829043421542946e-05 & lr-AdamW/pg2‚5.829043421542946e-05 trainer/global_step‚5399 $ _timestamp‚1.788394196238799e+09´X)Èä2²- _wandbruntime‚4951  _runtime‚4951ñDá7ë'³åO ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4417736946996 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚530 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚362691406839808 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚367133729740800 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚362691389800448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚370261995701760 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 _runtime‚4951.996102425 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚365593996947968 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 115031858 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚56575016 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚92376982 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚367133729740800 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 115031858 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚533 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4668232491008 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4442536119296 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚92376980 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚28945029 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4667998753792 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚358354072272180 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4417544950456 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28944499 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4442536119296 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 115054456 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚358353863236280 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4442339940352 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4442339940352 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚85519515 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚370262254342656 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚362691406839808 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 207408838 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚92376980 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚5399 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚56575019 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚353936318285824 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 _step‚437 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 207431438 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚85520048 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚367133942959104 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚367133729740800 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚362691389800448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚370261995701760 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 _runtime‚4951.996102425 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 258640896 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚365593996947968 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 115031858 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚56575016 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚92376982 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚367133729740800 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 115031858 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚533 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4668232491008 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4442536119296 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚92376980 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚28945029 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4667998753792 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚358354072272180 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚hì3‡  4417544950456 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚28944499 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4442536119296 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 115054456 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚358353863236280 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4442339940352 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4442339940352 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚85519515 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚370262254342656 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568µÓôÈ2µ- _wandbruntime‚4952  _runtime‚4952|cì§ö&¶ðM _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚365659522444288 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚362756445876736 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4443138331136 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 207468856 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚85534632 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚365659547347968 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚353999917670400 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚56585050 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4418530830152 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚358418465539912 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4669074897408 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚370328353132032 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚367199780400128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚530 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4443138331136 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4418338833404 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚362756428837376 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 207468856 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚85534632 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚365659547347968 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚353999917670400 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚56585050 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4418530830152 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚358418465539912 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4669074897408 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚370328353132032 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚367199780400128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚530 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4443138331136 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4418338833404 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚362756428837376 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_4_of_5‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3075774908065796 particle/accuracy_2_of_3‚1 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9919141530990601 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4302545487880707 jet/accuracy_6_of_6‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.1472751498222351 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4191831052303314 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.16672171652317047 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.43646442890167236 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3730441927909851 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 jet/accuracy_6_of_7‚0 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.05565507337450981 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3301455080509186 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +jet/accuracy_3_of_4‚0.06319252401590347 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 jet/accuracy_3_of_7‚0 _step‚440 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.028241649270057678 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_2‚0.7199650406837463 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3440665304660797 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3703960180282593 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.4299514889717102 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.013279268518090248 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2154761701822281 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42410722374916077 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 +particle/specificity‚0.9532360434532166 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3222222328186035 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4434826076030731 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 9"Purity/1FRt*SRqqt*FBt/event_purity‚0.0830819234251976 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.19752520322799683 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 +jet/accuracy_4_of_6‚0.07894736528396606 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.20021766424179077 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15103046596050262 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.31668585538864136 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.4352063238620758 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4376629889011383 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12462878972291946 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.34795188903808594 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24966934323310852 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.47644931077957153 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 _runtime‚5028.099639142 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.33741307258605957 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 *jet/accuracy_2_of_3‚0.2322157919406891 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4505678117275238 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09913796186447144 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.009188291616737843 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.44997438788414 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.49382713437080383 particle/accuracy_2_of_4‚1 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.04296296462416649 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.48898807168006897 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 6 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.422677606344223 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4124999940395355 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 particle/accuracy_4_of_6‚1 jet/accuracy_2_of_7‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4398705065250397 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3423669934272766 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08210155367851257 ,jet/accuracy_3_of_3‚0.021869206801056862 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5952380895614624 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.29938483238220215 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0021601382177323103 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.35547879338264465 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,jet/accuracy_4_of_5‚0.004962779115885496 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12809136509895325 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4537211060523987 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.644444465637207 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_5_of_6‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40835005044937134 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.42444440722465515 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.12615741789340973 *jet/accuracy_1_of_5‚0.9416534900665283 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9891853332519531 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39566051959991455 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16762672364711761 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_7‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.42951157689094543 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16460713744163513 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19391418993473053 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2621815502643585 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3478836417198181 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24823230504989624 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.40116962790489197 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9940556883811951 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2748143672943115 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2021273523569107 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21212123334407806 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994300901889801 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.17152926325798035 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.47704339027404785 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4515497386455536 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3828125 particle/accuracy_3_of_6‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 particle/accuracy_2_of_2‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4033828377723694 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 8"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.449436753988266 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15736918151378632 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.22402900457382202 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19786855578422546 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.009842673316597939 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.21917860209941864 6validation_average_jet_accuracy‚0.4149675965309143 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 % _timestamp‚1.7883942723416111e+09 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1481051743030548 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.65625 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.48874810338020325 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16500437259674072 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.005818032193928957 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6000000238418579 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.16026045382022858 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_4_of_7‚0 jet/accuracy_7_of_7‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 particle/accuracy_5_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6763941645622253 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9811828136444092 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.34146827459335327 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2044556736946106 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5081596374511719 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.02346918359398842 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4416043758392334 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6465567350387573 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.37358686327934265 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3738887906074524 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 particle/accuracy_7_of_7‚0.5 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.17500001192092896 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_2‚0.1041196882724762 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 (particle/accuracy‚0.8567970991134644 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 )loss/val_total_loss‚42.28424615348328 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 7"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44779172539711 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3331986665725708 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.21839478611946106 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.19674837589263916 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9933123588562012 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4971918761730194 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.49572303891181946 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4794864058494568 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.13271605968475342 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 particle/accuracy_4_of_7‚1 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23292098939418793 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18912044167518616 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3694637715816498 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.09940475970506668 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4480726420879364 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.1573483645915985 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.22495239973068237 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 *jet/accuracy_1_of_3‚0.8477792143821716 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5213783383369446 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8169935345649719 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1829911470413208 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3959892690181732 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.46666669845581055 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3773844242095947 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.16706125438213348 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6470588445663452 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9988245964050293 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1591976433992386 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9881480932235718 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_2‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004369402304291725 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.23148149251937866 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10600703954696655 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.003848330583423376 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4754047095775604 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9865649342536926 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1649370640516281 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.22846318781375885 *jet/accuracy_1_of_4‚0.8949350714683533 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02595452591776848 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.03333333507180214 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 *particle/sensitivity‚0.335767924785614 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9917521476745605 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.46712908148765564 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.41752347350120544 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9931535124778748 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.49427250027656555 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4480726420879364 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.05970899760723114 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02146678976714611 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02282273955643177 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.4402236044406891 ,"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.375 particle/accuracy_1_of_6‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.47860202193260193 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988713264465332 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.00358422938734293 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.48052719235420227 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.46732810139656067 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.012763580307364464 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06960449367761612 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13189321756362915 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15103046596050262 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_6‚0.4736842215061188 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6071428656578064 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.20252002775669098 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1433333456516266 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4183594882488251 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4610176384449005 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4574487507343292 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.1623208373785019 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.11764705926179886 particle/accuracy_1_of_1‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 trainer/global_step‚5435 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9922439455986023 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.45078274607658386 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14406488835811615 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 particle/accuracy_1_of_5‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20517253875732422 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1507575809955597 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2234269380569458 8"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.441497802734375 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.21735790371894836 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.45773810148239136 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 (particle/f_score‚0.42270785570144653 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.23415866494178772 7 Purity/*FRt2SRqqt*FBt/FBt_purit²½1[Zy‚0.9898380637168884 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09597700834274292 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5952380895614624 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4739583432674408 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9899560213088989 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4708809554576874 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4744444489479065 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.24473701417446136 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1879800409078598 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.034216590225696564 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12318841367959976 particle/accuracy_3_of_7‚1 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5214901566505432 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.15525656938552856 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.17034021019935608 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4166666567325592 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27306821942329407 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.003225806402042508 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.4035087823867798 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1634422093629837 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6538461446762085 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_1_of_6‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.44209328293800354 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 epoch‚8 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4503849744796753 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.06768275052309036 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4843134582042694 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_5_of_7‚0 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.4138311743736267 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9901374578475952 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9851189851760864 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3423616290092468 4 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920614361763 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36007484793663025 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.45845291018486023 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.29554104804992676 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 particle/accuracy_2_of_6‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41264525055885315 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276Aí¦˜gÇ®§ :"Purity/0FRt1SRqqt*FBt/event_purity‚0.37358686327934265 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3738887906074524 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 particle/accuracy_7_of_7‚0.5 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.17500001192092896 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_2‚0.1041196882724762 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 (particle/accuracy‚0.8567970991134644 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 )loss/val_total_loss‚42.28424615348328 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 7"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44779172539711 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3331986665725708 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.21839478611946106 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.19674837589263916 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9933123588562012 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4971918761730194 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.49572303891181946 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4794864058494568 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.13271605968475342 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 particle/accuracy_4_of_7‚1 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23292098939418793 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18912044167518616 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3694637715816498 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.09940475970506668 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4480726420879364 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.1573483645915985 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.22495239973068237 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 *jet/accuracy_1_of_3‚0.8477792143821716 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5213783383369446 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8169935345649719 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1829911470413208 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3959892690181732 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.46666669845581055 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3773844242095947 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.16706125438213348 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6470588445663452 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9988245964050293 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1591976433992386 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9881480932235718 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_2‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004369402304291725 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.23148149251937866 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10600703954696655 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.003848330583423376 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4754047095775604 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9865649342536926 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1649370640516281 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.22846318781375885 *jet/accuracy_1_of_4‚0.8949350714683533 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02595452591776848 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.03333333507180214 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 *particle/sensitivity‚0.335767924785614 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9917521476745605 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.46712908148765564 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.41752347350120544 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9931535124778748 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.49427250027656555 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4480726420879364 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.05970899760723114 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.02146678976714611 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02282273955643177 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.4402236044406891 ,"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.375 particle/accuracy_1_of_6‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.47860202193260193 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988713264465332 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.00358422938734293 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.48052719235420227 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.46732810139656067 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.012763580307364464 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06960449367761612 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13189321756362915 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15103046596050262 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_6‚0.4736842215061188 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6071428656578064 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.20252002775669098 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1433333456516266 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4183594882488251 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4610176384449005 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4574487507343292 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.1623208373785019 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.11764705926179886 particle/accuracy_1_of_1‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,"Purity/1FRt*SRqqt3FBt/event_purity‚0.125 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 trainer/global_step‚5435 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9922439455986023 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.45078274607658386 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14406488835811615 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9780701994895935 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 particle/accuracy_1_of_5‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20517253875732422 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1507575809955597 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1428571492433548 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2234269380569458 8"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.441497802734375 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.21735790371894836 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.45773810148239136 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 (particle/f_score‚0.42270785570144653 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.23415866494178772 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9898380637168884 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09597700834274292 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5952380895614624 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4739583432674408 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9899560213088989 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4708809554576874 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4744444489479065 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.24473701417446136 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1879800409078598 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.034216590225696564 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12318841367959976 particle/accuracy_3_of_7‚1 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5214901566505432 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.15525656938552856 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.17034021019935608 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4166666567325592 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27306821942329407 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.003225806402042508 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.4035087823867798 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1634422093629837 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6538461446762085 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_1_of_6‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.44209328293800354 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 epoch‚8 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4503849744796753 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.06768275052309036 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4843134582042694 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_5_of_7‚0 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.4138311743736267 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9901374578475952 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9851189851760864 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3423616290092468 4 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920614361763 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36007484793663025 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.45845291018486023 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.29554104804992676 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 particle/accuracy_2_of_6‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41264525055885315 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_4_of_5‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3075774908065796 particle/accuracy_2_of_3‚1 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9919141530990601 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4302545487880707 jet/accuracy_6_of_6‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.1472751498222351 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4191831052303314 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.16672171652317047 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.43646442890167236 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3730441927909851 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 jet/accuracy_6_of_7‚0 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.05565507337450981 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3301455080509186 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +jet/accuracy_3_of_4‚0.06319252401590347 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 jet/accuracy_3_of_7‚0 _step‚440 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.028241649270057678 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 *jet/accuracy_1_of_2‚0.7199650406837463 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3440665304660797 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3703960180282593 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.4299514889717102 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.013279268518090248 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2154761701822281 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42410722374916077 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 +particle/specificity‚0.9532360434532166 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3222222328186035 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4434826076030731 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 9"Purity/1FRt*SRqqt*FBt/event_purity‚0.0830819234251976 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.19752520322799683 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 +jet/accuracy_4_of_6‚0.07894736528396606 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.20021766424179077 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15103046596050262 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.31668585538864136 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.4352063238620758 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4376629889011383 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 :J¯¯A,"Purity/1FRt0SRqqt2FBt/event_purity‚0.12462878972291946 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.34795188903808594 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24966934323310852 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.47644931077957153 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 _runtime‚5028.099639142 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.33741307258605957 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 *jet/accuracy_2_of_3‚0.2322157919406891 6 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.942307710647583 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4505678117275238 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09913796186447144 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.009188291616737843 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.44997438788414 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.49382713437080383 particle/accuracy_2_of_4‚1 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.04296296462416649 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.48898807168006897 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 6 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.422677606344223 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4124999940395355 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 particle/accuracy_4_of_6‚1 jet/accuracy_2_of_7‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4398705065250397 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3423669934272766 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08210155367851257 ,jet/accuracy_3_of_3‚0.021869206801056862 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5952380895614624 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.29938483238220215 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0021601382177323103 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.35547879338264465 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,jet/accuracy_4_of_5‚0.004962779115885496 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12809136509895325 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4537211060523987 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.644444465637207 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_5_of_6‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40835005044937134 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.42444440722465515 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.12615741789340973 *jet/accuracy_1_of_5‚0.9416534900665283 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9891853332519531 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39566051959991455 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16762672364711761 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_7‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.42951157689094543 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16460713744163513 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19391418993473053 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2621815502643585 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3478836417198181 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24823230504989624 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.40116962790489197 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9940556883811951 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2748143672943115 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2021273523569107 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21212123334407806 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994300901889801 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.17152926325798035 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.47704339027404785 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4515497386455536 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3828125 particle/accuracy_3_of_6‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 particle/accuracy_2_of_2‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4033828377723694 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 8"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.449436753988266 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15736918151378632 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.22402900457382202 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19786855578422546 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.009842673316597939 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.21917860209941864 6validation_average_jet_accuracy‚0.4149675965309143 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 % _timestamp‚1.7883942723416111e+09 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1481051743030548 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.65625 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.48874810338020325 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16500437259674072 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.005818032193928957 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6000000238418579 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.16026045382022858 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_4_of_7‚0 jet/accuracy_7_of_7‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 particle/accuracy_5_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6763941645622253 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9811828136444092 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.34146827459335327 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2044556736946106 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5081596374511719 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.02346918359398842 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4416043758392334 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6465567350387573¸šÉ˜¹’È‚(:‡ ¬öâÔ¾˜Íu gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚135.303' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.0606 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Tþ/ÔOÉ‚(:Ä ¬öâÔºÙÚx network.sent‚ 1.1367126e+07 disk.sda2.out‚0 disk.rbd3.out‚89.62451171875 cpu‚25.704491498773027 disk.sda3.in‚0.125 disk.sda1.in‚0 disk.rbd1.in‚0 disk.rbd2.in‚0& disk.nvme0n1.out‚51145.52783203125 disk.rbd3.in‚0) proc.memory.availableMB‚ 1146.33984375% disk./.usageGB‚1065.8863258361816 disk.dm-0.in‚ 265.6328125 disk.dm-1.in‚0.125 disk.sda1.out‚0 network.recv‚ 1.734606e+06) disk./.usagePercent‚71.53252566506252! disk.rbd2.out‚424.10009765625 disk.nvme0n1.in‚ 265.6328125 disk.drbd1777.in‚0 disk.sda.in‚0.125 disk.sda.out‚ 1620.578125 disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.in‚ 265.6328125 disk.dm-1.out‚ 1620.578125 disk.sda3.out‚ 1620.578125 disk.rbd1.out‚ 2.6015625 disk.rbd0.in‚0 disk.drbd1777.out‚0! proc.memory.rssMB‚ 2894.578125* proc.memory.percent‚17.667102813720703( disk.nvme0c0n1.out‚51145.52783203125 disk.sda2.in‚0# disk.dm-0.out‚51145.52783203125$ memory_percent‚93.00329685211182 proc.cpu.threads‚17ææ 2Ê- _wandbruntime‚5042  _runtime‚50429°ÕïÈË trainer/global_step‚5449 $ _timestamp‚1.788394286420436e+09 _runtime‚5042.176430589 _step‚441 ' lr-AdamW/pg1‚4.9636640059512726e-05 ' lr-AdamW/pg2‚4.9636640059512726e-05dŠõÍÌÇ _step‚441 ' lr-AdamW/pg1‚4.9636640059512726e-05 ' lr-AdamW/pg2‚4.9636640059512726e-05 trainer/global_step‚5449 $ _timestamp‚1.788394286420436e+09 _runtime‚5042.176430589¹ÆŒ2Í- _wandbruntime‚5042  _runtime‚5042gë'ÎåO E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4473198327074 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚371873587654144 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚375062973273600 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚375062704146944 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚57374691 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 209945427 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 116324845 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚371873800872448 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚367375337509888 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚29327541 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4473390323614 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 trainer/global_step‚5449 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚362899536756638 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚358426146433024 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _step‚442 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4498267183616 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚511 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 209922827 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚367375320470528 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚29327033 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚86701721 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 _runtime‚ 5042.17761456 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚93597982 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚508 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 % _timestamp‚1.7883942864214165e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚367375320470528 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚86702232 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚93597984 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚362899327720738 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚93597984 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4498463362560Û±$uß#ÏêO VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 269126656 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚362899536756638 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚358426146433024 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 _step‚442 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4498267183616 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚511 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 244222976 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 209922827 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚367375320470528 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚29327033 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚86701721 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 _runtime‚ 5042.17761456 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚93597982 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚508 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 % _timestamp‚1.7883942864214165e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚367375320470528 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚86702232 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚93597984 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚362899327720738 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚93597984 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4498463362560 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4473198327074 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚371873587654144 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚375062973273600 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚375062704146944 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚57374691 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 209945427 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.fOU4åreed‚0 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 116324845 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚371873800872448 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚367375337509888 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚29327541 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4473390323614 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 trainer/global_step‚5449ºâîö2Ð- _wandbruntime‚5042  _runtime‚5042äZ‰Xõ&ÑïM PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4473992210022 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚371939638313472 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚371939638313472 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚510 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚86717333 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4727261698560 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚367440376546816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚370402071810048 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 209960219 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚513 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚371939425081856 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 209982845 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚5042.936518935 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4499065574400 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚86716820 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 _step‚443 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4474184206770 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚367440376546816 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚93614504 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 116368341 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 116345717 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚7503þLõú&ÒôM B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚86717333 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4727261698560 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚367440376546816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚370402071810048 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 209960219 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚513 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚371939425081856 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 209982845 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚5042.936518935 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4499065574400 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚86716820 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 _step‚443 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4474184206770 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚367440376546816 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚93614504 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 116368341 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 116345717 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4473992210022 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚371939638313472 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚371939638313472 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚510 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0»–™æ¼2Ó- _wandbruntime‚5042  _runtime‚5042„ka@YÔÓ 0loss/SRqqt4/detection_loss‚2.591595492192677 _runtime‚5042.937684171 0loss/SRqqt1/detection_loss‚1.782238271122887 1loss/SRqqt3/detection_loss‚1.9019143464083368 0loss/FRt4/assignment_loss‚3.7086637918628864 1loss/SRqqt1/assignment_loss‚1.566201253448214 2loss/SRqqt3/assignment_loss‚1.5097614469284073 %loss/total_loss‚43.07466464366789 epoch‚9 % _timestamp‚1.7883942871819096e+09 2loss/SRqqt2/assignment_loss‚1.2905535341308612 .loss/FRt3/detection_loss‚2.661838917038109 .loss/FBt3/detection_loss‚2.169031488043921 trainer/global_step‚5449 /loss/FRt1/detection_loss‚2.2230244233291976 /loss/FRt2/detection_loss‚1.6382978615615893 -loss/FRt3/assignment_loss‚3.1207842426964 /loss/FBt1/detection_loss‚3.5239169359207154 1loss/SRqqt2/detection_loss‚1.0110332065723775 .loss/FBt4/detection_loss‚2.470289421081543 0loss/FRt2/assignment_loss‚1.4621260674721603 2loss/SRqqt4/assignment_loss‚1.7178465390710957 0loss/FRt1/assignment_loss‚2.8369588930996095 1loss/FBt2/assignment_loss‚0.12189030308951164 1loss/FBt3/assignment_loss‚0.27515642230753745 .loss/FRt4/detection_loss‚2.589560578118509 /loss/FBt2/detection_loss‚0.6526098029809237 _step‚444 1loss/FBt1/assignment_loss‚0.05049123189610327 1loss/FBt4/assignment_loss‚0.19888017329431024‡¾®À^ÕØ 1loss/FBt4/assignment_loss‚0.19888017329431024 0loss/SRqqt4/detection_loss‚2.591595492192677 _runtime‚5042.937684171 0loss/SRqqt1/detection_loss‚1.782238271122887 1loss/SRqqt3/detection_loss‚1.9019143464083368 0loss/FRt4/assignment_loss‚3.7086637918628864 1loss/SRqqt1/assignment_loss‚1.566201253448214 2loss/SRqqt3/assignment_loss‚1.5097614469284073 %loss/total_loss‚43.07466464366789 epoch‚9 % _timestamp‚1.7883942871819096e+09 2loss/SRqqt2/assignment_loss‚1.2905535341308612 .loss/FRt3/detection_loss‚2.661838917038109 .loss/FBt3/detection_loss‚2.169031488043921 trainer/global_step‚5449 /loss/FRt1/detection_loss‚2.2230244233291976 /loss/FRt2/detection_loss‚1.6382978615615893 -loss/FRt3/assignment_loss‚3.1207842426964 /loss/FBt1/detection_loss‚3.5239169359207154 1loss/SRqqt2/detection_loss‚1.0110332065723775 .loss/FBt4/detection_loss‚2.470289421081543 0loss/FRt2/assignment_loss‚1.4621260674721603 2loss/SRqqt4/assignment_loss‚1.7178465390710957 0loss/FRt1/assignment_loss‚2.8369588930996095 1loss/FBt2/assignment_loss‚0.12189030308951164 1loss/FBt3/assignment_loss‚0.27515642230753745 .loss/FRt4/detection_loss‚2.589560578118509 /loss/FBt2/detection_loss‚0.6526098029809237 _step‚444 1loss/FBt1/assignment_loss‚0.05049123189610327¼Tè+Ö‚(:’ »öâÔ¤þÞu gpu.0.gpu‚21.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚157.096' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.419200000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0kÆNׂ(:à »öâÔ⯸x& disk.nvme0n1.out‚51243.82080078125 disk.sda.in‚0.125 disk.rbd0.in‚0* proc.memory.percent‚17.667102813720703 disk.rbd1.in‚0 disk.dm-1.out‚ 1621.421875 network.recv‚ 1.741366e+06) disk./.usagePercent‚71.53252566506252 disk.dm-0.in‚ 265.6328125# disk.dm-0.out‚51243.82080078125 disk.sda3.out‚ 1621.421875 disk.drbd1777.out‚0# memory_percent‚93.0034875869751 network.sent‚ 1.1470874e+07( disk.nvme0c0n1.out‚51243.82080078125 disk.rbd3.in‚0! disk.rbd2.out‚424.24072265625 disk.nvme0n1.in‚ 265.6328125 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0 disk.rbd3.out‚89.63232421875 disk.sda.out‚ 1621.421875 disk.rbd0.out‚ 0.2978515625) proc.memory.availableMB‚ 1146.30859375 disk.dm-1.in‚0.125! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.in‚0 disk.sda3.in‚0.125! proc.memory.rssMB‚ 2894.578125 cpu‚25.704335746800517 proc.cpu.threads‚17% disk./.usageGB‚1065.8863258361816 disk.rbd1.out‚ 2.6640625 disk.rbd2.in‚0 disk.sda1.out‚0P¨{ÛØ‚(: ÊöâÔþ¬¹u gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚152.8' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.560000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ù¼cËMÙ‚(: ÊöâÔòèËw disk.dm-0.in‚ 265.6328125 disk.sda2.in‚0 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd1.out‚ 2.6640625) disk./.usagePercent‚71.53252412901732 disk.rbd3.in‚0 disk.sda3.in‚0.125 disk.sda3.out‚ 1638.55859375 disk.sda2.out‚0! proc.memory.rssMB‚ 2894.578125 disk.sda.out‚ 1638.55859375 disk.sda1.in‚0* proc.memory.percent‚17.667102813720703 network.sent‚ 1.1557329e+07 disk.rbd1.in‚0 disk.sda1.out‚0 disk.drbd1777.in‚0# memory_percent‚93.0034875869751 network.recv‚ 1.746562e+06 disk.dm-1.in‚0.125 disk.sda.in‚0.125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625& disk.nvme0c0n1.out‚51245.814453125 disk.rbd3.out‚89.69482421875 disk.dm-1.out‚ 1638.55859375 disk.rbd2.in‚0 cpu‚25.704090388067925$ disk./.usageGB‚1065.886302947998! disk.dm-0.out‚51245.814453125! disk.rbd2.out‚425.77978515625 disk.drbd1777.out‚0) proc.memory.availableMB‚ 1146.30859375 disk.nvme0n1.in‚ 265.6328125$ disk.nvme0n1.out‚51245.814453125º¡§d2Ú- _wandbruntime‚5084  _runtime‚5084\»jÈÛ _step‚445 ' lr-AdamW/pg1‚4.1658855500510426e-05 ' lr-AdamW/pg2‚4.1658855500510426e-05 trainer/global_step‚5499 $ _timestamp‚1.788394328815089e+09 _runtime‚5084.571157404WØ{§ÍÜÇ ' lr-AdamW/pg1‚4.1658855500510426e-05 ' lr-AdamW/pg2‚4.1658855500510426e-05 trainer/global_step‚5499 $ _timestamp‚1.788394328815089e+09 _runtime‚5084.571157404 _step‚445½A`Í~2Ý- _wandbruntime‚5084  _runtime‚5084^ϺÁ©ÞæO TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚94423984 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 117394395 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚375165486867968 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4538214551040 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚87459605 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚94423982 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚94423982 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4538410729984 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚57876388 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚29583217 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 211795779 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚375165486867968 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4768159953920 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540›F<ñ'ßëO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 % _timestamp‚1.7883943288162336e+09 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚375165700086272 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚378380956179968 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 117371797 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚370627272316928 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4538410729984 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚57876391 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚29583757 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4768402079744 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚370627272316928 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚94423982 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4538410729984 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚57876388 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚29583217 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 211795779 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚375165486867968 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4768159953920 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚94423984 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 117394395 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚375165486867968 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4538214551040 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚87459605 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚94423982 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚94440502 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 117415293 trainer/global_step‚5499 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚87474743 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚361669715046400 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4513713529430 % _timestamp‚1.7883943295835187e+09 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4513905526178 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4539209134080 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4539209134080 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 211833171 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚57886422 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 211833171 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚29588833 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _runtime‚5085.340163087 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚366183411536470 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚375231537527296 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _step‚447 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚370692311353856 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4539012941824 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4539012941824 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚378447584722432 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚373678346626048 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚373678321722368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚370692328393216 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚57886425 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚94440502 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 211855797 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚366183620572578 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚375231537527296 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 117392669 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚370692328393216 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0— †äõM K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚87474743 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚361669715046400 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚512 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 258627584 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4513713529430 % _timestamp‚1.7883943295835187e+09 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 233723904 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4513905526178 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4539209134080 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4539209134080 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 211833171 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚57886422 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 211833171 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚29588833 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 _runtime‚5085.340163087 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚366183411536470 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚375231537527296 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 _step‚447 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚370692311353856 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4539012941824 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4539012941824 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚378447584722432 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚373678346626048 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚373678321722368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚370692328393216 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚57886425 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚94440502 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 211855797 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚366183620572578 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚375231537527296 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 117392669 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚370692328393216 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚94440504 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚94440502 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 117415293 trainer/global_step‚5499 BßXWé‚(:Ì èöâÔ®¢Øx disk.sda.in‚0.125( disk.nvme0c0n1.out‚51442.74755859375 disk.rbd3.out‚89.77294921875 disk.sda3.out‚ 1675.69140625 disk.drbd1777.out‚0$ memory_percent‚93.00875663757324! proc.memory.rssMB‚ 2894.578125 network.recv‚ 1.756516e+06 disk.dm-1.out‚ 1675.69140625 disk.nvme0n1.in‚ 265.6328125 disk.rbd0.out‚ 0.2978515625 disk.rbd3.in‚0# disk.dm-0.out‚51442.74755859375 proc.cpu.threads‚17) disk./.usagePercent‚71.53253334528854 disk.sda.out‚ 1675.69140625 disk.drbd1777.in‚0 disk.sda2.out‚0% disk./.usageGB‚1065.8864402770996 disk.rbd2.in‚0 disk.rbd0.in‚0 network.sent‚ 1.1639044e+07 disk.dm-1.in‚0.125 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 265.6328125( proc.memory.availableMB‚ 1145.4453125* proc.memory.percent‚17.667102813720703 disk.rbd1.in‚0 disk.rbd1.out‚ 2.6962890625& disk.nvme0n1.out‚51442.74755859375 disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125 cpu‚25.703863374622305! disk.rbd2.out‚427.45556640625£A€‘ê‚(:† ÷öâÔ²ÊÑu gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.375' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.675 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0½ýR*Jë‚(:¿ ÷öâÔ¹‹Õx disk.rbd0.in‚0 disk.rbd3.in‚0 disk.rbd2.in‚0 disk.sda2.in‚0 cpu‚25.7036610315301 disk.dm-1.in‚0.125 disk.dm-1.out‚ 1693.328125 disk.rbd1.out‚ 2.75927734375$ disk.nvme0n1.out‚51447.224609375 disk.sda1.in‚0 disk.drbd1777.in‚0 disk.sda2.out‚0 network.recv‚ 1.762577e+06 disk.sda3.out‚ 1693.328125 disk.rbd1.in‚0 proc.cpu.threads‚17! disk.dm-0.out‚51447.224609375 disk.sda.out‚ 1693.328125 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 265.6328125& disk.nvme0c0n1.out‚51447.224609375 network.sent‚ 1.1716529e+07) disk./.usagePercent‚71.53266672521387! disk.rbd2.out‚429.10791015625 disk.rbd3.out‚89.77294921875) proc.memory.availableMB‚ 1145.01953125* proc.memory.percent‚17.667102813720703$ disk./.usageGB‚1065.888427734375 disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125 disk.nvme0n1.in‚ 265.6328125$ memory_percent‚93.01135540008545! proc.memory.rssMB‚ 2894.578125 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.out‚0 disk.sda.in‚0.125’ß·˜2ì- _wandbruntime‚5127  _runtime‚5127*S­àÇíÁ & lr-AdamW/pg1‚3.436374143886349e-05 & lr-AdamW/pg2‚3.436374143886349e-05 trainer/global_step‚5549 % _timestamp‚1.7883943715057876e+09 _runtime‚5127.261907862 _step‚449˜úÜÌîÆ trainer/global_step‚5549 % _timestamp‚1.7883943715057876e+09 _runtime‚5127.261907862 _step‚449 & lr-AdamW/pg1‚3.436374143886349e-05 & lr-AdamW/pg2‚3.436374143886349e-05Áæ¨.Ò2ï- _wandbruntime‚5127  _runtime‚5127‚Xì'ðæO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚364786067851264 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 213691331 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚369338708817154 % _timestamp‚1.7883943715069964e+09 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚373879224163328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚378457386081792 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4810370658816 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚378457386081792 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 trainer/global_step‚5549 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚95249982 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚373879241202688 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4552832962430 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚381699441700864 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4810610687488 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚373879224163328 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚369338917853054 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚378457599300096 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚5127.263208373 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 118441347 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚376889071042048 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚373879241202688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚95249982 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4552640965890 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚95249984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 213668731 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 118418749 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚450 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4578161918464 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4578161918464 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚95249984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚513 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚58378088 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚381699706633216 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 118441347 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750—õyñ'ñëO M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚373879224163328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚378457386081792 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4810370658816 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚378457386081792 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 trainer/global_step‚5549 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚95249982 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚373879241202688 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4552832962430 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚381699441700864 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4810610687488 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚373879224163328 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚369338917853054 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚378457599300096 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚5127.263208373 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 118441347 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚376889071042048 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚373879241202688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚95249982 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4552640965890 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚95249984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 213668731 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 118418749 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚450 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4578161918464 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4578161918464 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚95249984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚513 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚58378088 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚381699706633216 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 118441347 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚364786067851264 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 213691331 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚369338708817154 % _timestamp‚1.7883943715069964e+09 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065©íýØ2ò- _wandbruntime‚5128  _runtime‚5128´t¥×bóðM S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚378523223509504 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 213706123 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚378523436741120 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚5549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚95266502 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚29847043 WIDeviceStatsMonitor.on_train_batch_<)‘é”end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883943722673185e+09 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 213728749 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚88235168 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4579156501504 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 118462245 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4578960309248 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚373944263200256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 213706123 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4578960309248 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 118462245 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚369403311120786 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚58388125 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4553434848838 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚373944263200256 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4811446638592 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚364849667235840 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚95266504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚381766068080640 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚88234652 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚516 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4579156501504 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚95266504 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚95266502 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4811210817536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4553626845586 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚381765807355904 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 213728749 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 118439621 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚376954596538368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚364849684275200 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚378523436741120 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 _runtime‚5128.023747636 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚373944280239616 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816`õû&ôõM _step‚451 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚369403311120786 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚58388125 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4553434848838 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚373944263200256 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4811446638592 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚364849667235840 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚95266504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚381766068080640 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚88234652 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚516 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4579156501504 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚95266504 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚95266502 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4811210817536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚513 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4553626845586 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚381765807355904 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 213728749 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 118439621 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚376954596538368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚364849684275200 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚378523436741120 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 _runtime‚5128.023747636 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚373944280239616 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚378523223509504 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 213706123 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚378523436741120 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 260724736 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 trainer/global_step‚5549 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚95266502 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚29847043 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883943722673185e+09 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 213728749 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚88235168 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4579156501504 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 118462245 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4578960309248 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚373944263200256 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 213706123 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4578960309248 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 118462245 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 215564283 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 119465701 trainer/global_step‚5599 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚377131176009728 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚5169.712038202 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚377131193049088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚381749498513920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚372558608401262 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30100538 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚96075984 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/requestedòhYk _bytes.all.current‚ 209035900 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 119488299 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚500 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚372558399365362 % _timestamp‚1.7883944139525435e+09 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚58879791 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚381749285295616 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30100038 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4618109285888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚58879788 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 _step‚454 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚377131193049088 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚503 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 215541683 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚377131176009728 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚381749285295616 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚385018137328640 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚380165370761728 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750»+ȧñ'ƒëO L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚96075982 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚380165345858048 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚96075984 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 119488299 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚500 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚372558399365362 % _timestamp‚1.7883944139525435e+09 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚58879791 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚381749285295616 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30100038 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4618109285888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚58879788 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 _step‚454 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚377131193049088 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚503 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 215541683 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚377131176009728 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚381749285295616 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚385018137328640 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚380165370761728 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚88980329 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4618109285888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4592362285298 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4618305464832 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚96075982 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 119465701 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 215564283 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 119465701 trainer/global_step‚5599 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚377131176009728 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 _runtime‚5169.712038202 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚377131193049088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚381749498513920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚372558608401262 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30100538 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚377196215046656 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚380230896258048 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚88995144 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚30105853 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚96092502 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚385084228988928 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚531 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚58889822 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 215579075 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 215601701 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚96092502 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚534 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚380230871354368 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 119486573 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚381815122723328 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4853357634560 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚58889825 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4593348164994 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚381815335954944 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚381815335954944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 _step‚455 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 215601701 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4853599747072 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚385084496005120 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚368029653504000 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824éN¬Y#†õM C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 trainer/global_step‚5599 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚96092504 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 119509197 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4618907676672 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚377196215046656 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚30105322 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚372623001668994 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 215601701 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚96092502 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚534 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚380230871354368 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 119486573 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚381815122723328 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4853357634560 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚58889825 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4593348164994 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚381815335954944 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚381815335954944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 _step‚455 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 215601701 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 267016192 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4853599747072 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚385084496005120 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚368029653504000 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚372622792632886 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚377196232086016 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚96092504 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚377196215046656 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚380230896258048 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚88995144 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚30105853 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚96092502 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚385084228988928 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OGDeviceSt)LW#¢atsMonitor.on_train_batch_end/inactive_split.small_pool.current‚531 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚58889822 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 215579075 PDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 217414635 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 217437235 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4894714104832 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚385041184509440 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚89739802 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4658056653312 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 120535251 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚375778089913570 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5649 % _timestamp‚1.7883944570549123e+09 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 120512653 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚380383144895488 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4658056653312 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚380383144895488;„ôvñ'•ëO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4894714104832 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚385041184509440 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚89739802 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4658056653312 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 120535251 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚375778089913570 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 trainer/global_step‚5649 % _timestamp‚1.7883944570549123e+09 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 120512653 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚380383144895488 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4658056653312 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚380383144895488 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚380383127856128 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4658252832256 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30358314 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚383441620674048 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 120512653 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4632275601246 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _step‚458 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚371146023348224 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4658252832256 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚59381488 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚388336359682560 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚385041184509440 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚96901982 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚96901984 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚96901984 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚383441645577728 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚371146006308864 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 217414635 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 217437235 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243ÊÖø*%2–- _wandbruntime‚5213  _runtime‚5213_+öê0 —ïM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 120556149 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚380448183932416 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚385107235168768 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 217474653 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚371209605693440 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4632877487654 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 250501120 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚375842692217202 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4658855044096 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚380448166893056 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4658855044096 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚59391525 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.pea 4Åk‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚96918502 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 trainer/global_step‚5649 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚385107021937152 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4659051236352 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 $ _timestamp‚1.788394457820118e+09 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚380448183932416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚388402724132352 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 275404800 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚89755463 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚555 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 217474653 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚89754908 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚371209622732800 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4633069484402 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0CÀ¡óú&˜ôM A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚385107235168768 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 217474653 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚371209605693440 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4632877487654 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 250501120 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚375842692217202 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4658855044096 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚380448166893056 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4658855044096 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚59391525 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚96918502 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 trainer/global_step‚5649 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚385107021937152 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4659051236352 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 $ _timestamp‚1.788394457820118e+09 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚380448183932416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚388402724132352 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 275404800 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚89755463 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚555 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 217474653 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚89754908 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚371209622732800 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4633069484402 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 120556149 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚380448183932416Ë3T§ž2™- _wandbruntime‚5213  _runtime‚5213‹%k~ašÛ /loss/FRt2/detection_loss‚1.9560548696279914 &loss/total_loss‚41.696948955989875 _runtime‚5213.577836323 1loss/FBt2/assignment_loss‚0.16069934924571008 0loss/SRqqt1/detection_loss‚1.555649102532438 1loss/SRqqt2/detection_loss‚0.8514334548422149 /loss/FBt4/detection_loss‚2.7693904588619866 % _timestamp‚1.7883944578219779e+09 0loss/FRt3/assignment_loss‚2.8700862336158752 1loss/FBt3/assignment_loss‚0.21204494647924868 /loss/FRt1/detection_loss‚2.1959356358076665 2loss/SRqqt3/assignment_loss‚1.3134496828707138 2loss/SRqqt4/assignment_loss‚2.0273709672323443 /loss/FRt3/detection_loss‚2.4807397969563802 -loss/FBt1/detection_loss‚2.41636298596859 /loss/FBt3/detection_loss‚2.4508595571798435 _step‚460 0loss/FRt4/assignment_loss‚3.9479308873414993 /loss/FRt4/detection_loss‚2.7575611333761896 trainer/global_step‚5649 0loss/FRt2/assignment_loss‚1.5033353234266307 2loss/SRqqt2/assignment_loss‚0.8368805628204324 0loss/FBt4/assignment_loss‚0.1197212891105058 0loss/FRt1/assignment_loss‚2.9822734708481646 1loss/SRqqt4/detection_loss‚2.4922645337748945 0loss/FBt2/detection_loss‚0.44798297789480007 epoch‚9 1loss/SRqqt3/detection_loss‚1.7261208995143966 2loss/SRqqt1/assignment_loss‚1.5172048058759953 1loss/FBt1/assignment_loss‚0.10559603078536384r1þf›à 2loss/SRqqt2/assignment_loss‚0.8368805628204324 0loss/FBt4/assignment_loss‚0.1197212891105058 0loss/FRt1/assignment_loss‚2.9822734708481646 1loss/SRqqt4/detection_loss‚2.4922645337748945 0loss/FBt2/detection_loss‚0.44798297789480007 epoch‚9 1loss/SRqqt3/detection_loss‚1.7261208995143966 2loss/SRqqt1/assignment_loss‚1.5172048058759953 1loss/FBt1/assignment_loss‚0.10559603078536384 /loss/FRt2/detection_loss‚1.9560548696279914 &loss/total_loss‚41.696948955989875 _runtime‚5213.577836323 1loss/FBt2/assignment_loss‚0.16069934924571008 0loss/SRqqt1/detection_loss‚1.555649102532438 1loss/SRqqt2/detection_loss‚0.8514334548422149 /loss/FBt4/detection_loss‚2.7693904588619866 % _timestamp‚1.7883944578219779e+09 0loss/FRt3/assignment_loss‚2.8700862336158752 1loss/FBt3/assignment_loss‚0.21204494647924868 /loss/FRt1/detection_loss‚2.1959356358076665 2loss/SRqqt3/assignment_loss‚1.3134496828707138 2loss/SRqqt4/assignment_loss‚2.0273709672323443 /loss/FRt3/detection_loss‚2.4807397969563802 -loss/FBt1/detection_loss‚2.41636298596859 /loss/FBt3/detection_loss‚2.4508595571798435 _step‚460 0loss/FRt4/assignment_loss‚3.9479308873414993 /loss/FRt4/detection_loss‚2.7575611333761896 trainer/global_step‚5649 0loss/FRt2/assignment_loss‚1.5033353234266307Ì^£Ðªœ‚(:’ à÷âÔ£¬Öu gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.895' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.979000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0}Zé$N‚(:à à÷âԛȳx disk.sda1.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 1770.5078125$ memory_percent‚93.01636219024658 disk.sda.out‚ 1770.5078125! disk.dm-0.out‚51941.865234375 disk.dm-1.in‚0.125 disk.rbd1.out‚ 2.822265625 proc.cpu.threads‚17$ disk./.usageGB‚1065.891487121582 disk.rbd3.out‚89.89892578125 disk.sda3.in‚0.125 cpu‚25.70255837455719 disk.drbd1777.in‚0 disk.sda.in‚0.125! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd2.in‚0) disk./.usagePercent‚71.53287204325635 disk.rbd0.in‚0 disk.rbd1.in‚0! disk.rbd2.out‚435.63916015625* proc.memory.percent‚17.667126655578613 disk.drbd1777.out‚0 network.recv‚ 1.794342e+06 disk.sda1.out‚0 disk.rbd0.out‚ 0.2978515625 disk.dm-1.out‚ 1770.5078125& disk.nvme0c0n1.out‚51941.865234375 disk.sda2.in‚0# proc.memory.rssMB‚ 2894.58203125 network.sent‚ 1.1919791e+07 disk.nvme0n1.in‚ 265.6328125$ disk.nvme0n1.out‚51941.865234375 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125) proc.memory.availableMB‚ 1144.19921875¹#’ž‚(:‡ ï÷âÔ÷ÎËu gpu.0.gpu‚54.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.779' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.3558 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0>LŸ‚(:Á ï÷âÔ›ë÷w disk.rbd1.out‚ 2.822265625 disk.drbd1777.out‚0% disk./.usageGB‚1065.8914680480957 disk.sda2.in‚0! disk.dm-0.out‚51942.208984375! disk.rbd2.out‚435.74072265625 disk.drbd1777.in‚0$ memory_percent‚93.01788806915283* proc.memory.percent‚17.667126655578613 cpu‚25.70246222736862) disk./.usagePercent‚71.53287076321867 disk.sda.in‚0.125 disk.rbd0.in‚0# proc.memory.rssMB‚ 2894.58203125 disk.sda3.out‚ 1771.265625 disk.nvme0n1.in‚ 265.6328125$ disk.nvme0n1.out‚51942.208984375 disk.sda1.out‚0 proc.cpu.threads‚17 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125 disk.dm-1.in‚0.125 disk.dm-1.out‚ 1771.265625 network.recv‚ 1.800599e+06 disk.rbd1.in‚0 disk.sda1.in‚0) proc.memory.availableMB‚ 1143.94921875 network.sent‚ 1.2007748e+07 disk.rbd3.out‚89.89892578125 disk.sda3.in‚0.125 disk.rbd2.in‚0 disk.sda.out‚ 1771.265625 disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.in‚ 265.6328125& disk.nvme0c0n1.out‚51942.208984375 disk.sda2.out‚0ƒÔyœ ‚(:‘ þ÷âÔ„Ìu gpu.0.gpu‚58.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚93.466' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.693199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0€\¶>¡‚(:³ þ÷âÔ§Íýw! disk.rbd2.out‚437.25244140625 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.rbd2.in‚0% disk.nvme0n1.out‚52040.6259765625 disk.drbd1777.out‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda.in‚0.125 disk.sda.out‚1788.25 disk.sda1.in‚0' disk.nvme0c0n1.out‚52040.6259765625 disk.sda2.in‚0" disk.dm-0.out‚52040.6259765625 disk.dm-1.in‚0.125 disk.rbd1.out‚ 2.822265625 disk.rbd0.out‚ 0.2978515625) disk./.usagePercent‚71.53287204325635 disk.sda1.out‚0% proc.memory.availableMB‚ 1144.1875 cpu‚25.702361496525032 proc.cpu.threads‚17 disk.dm-0.in‚ 265.6328125 disk.rbd0.in‚0 network.sent‚ 1.2013056e+07$ disk./.usageGB‚1065.891487121582 disk.sda2.out‚0 disk.sda3.in‚0.125 disk.dm-1.out‚1788.25$ memory_percent‚93.01643371582031# proc.memory.rssMB‚ 2894.58203125 network.recv‚ 1.80456e+06 disk.rbd3.out‚89.89892578125* proc.memory.percent‚17.667126655578613 disk.rbd3.in‚0 disk.sda3.out‚1788.25 disk.rbd1.in‚0ÀÕ½2¢- _wandbruntime‚5255  _runtime‚5255³S!Ìɣà ' lr-AdamW/pg2‚1.6632451467187788e-05 trainer/global_step‚5699 % _timestamp‚1.7883944992874033e+09 _runtime‚5255.043858111 _step‚461 ' lr-AdamW/pg1‚1.6632451467187788e-05¨^~aÎ¤È _step‚461 ' lr-AdamW/pg1‚1.6632451467187788e-05 ' lr-AdamW/pg2‚1.6632451467187788e-05 trainer/global_step‚5699 % _timestamp‚1.7883944992874033e+09 _runtime‚5255.043858111Í;×2¥- _wandbruntime‚5255  _runtime‚5255µ$Êe: ¦æO WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30617283 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚391654660260864 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 121582203 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4936499838464 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4671804924114 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 121582203 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚391654395328512 % _timestamp‚1.7883944992881155e+09 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚378997989497678 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚388333083723264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚97727984 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚388333296941568 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚383635096741888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚508 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚386717920393728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚97727982 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚374325992577024 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚388333083723264 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚97727984 TGDeviceStatsMonitor.on> ײ_train_batch_start/inactive_split.small_pool.freed‚30616775 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚5255.044271359 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚59883188 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚511 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 219287587 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4698004020736 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚90499963 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0Ú·'ñ'§ëO C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚391654395328512 % _timestamp‚1.7883944992881155e+09 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚378997989497678 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚388333083723264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚97727984 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚388333296941568 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚383635096741888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚508 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚386717920393728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚97727982 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚374325992577024 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚388333083723264 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚97727984 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30616775 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚5255.044271359 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚59883188 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚511 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 219287587 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4698004020736 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚90499963 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30617283 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚391654660260864 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 121582203 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4936499838464 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4671804924114 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 121582203 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581ιà›x2¨- _wandbruntime‚5255  _runtime‚5255 zQô&©îM OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 121603101 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 _step‚463 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚388399134382592 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4698802411520 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚516 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚97744504 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 121580477 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚97744502 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚374389574922240 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4672790803810 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚383700135778816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚5699 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚59893222 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚391721021577216 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4937575687168 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚383700118739456 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 # _timestamp‚1.78839450004455e+09 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚90514980 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚97744502 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚388398921150976 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 121603101 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚379062173729302 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚97744504 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4698998603776 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4698802411520 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 121580477 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚383700118739456 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 219347605 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚5255.801144899 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 219324979 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚383700135778816 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚5699 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚59893222 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚391721021577216 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4937575687168 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚383700118739456 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 # _timestamp‚1.78839450004455e+09 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚90514980 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚97744502 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚388398921150976 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 121603101 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚379062173729302 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚97744504 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4698998603776 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4698802411520 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 121580477 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚383700118739456 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 219347605 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚5255.801144899 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 219324979 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 _step‚463 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚388399134382592 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4698802411520 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚516 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚97744504 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 121580477 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚97744502 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 242112512 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚374389574922240 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624ÏÍ—22«- _wandbruntime‚5255  _runtime‚5255ƒÀ FÛ¬× /loss/FBt4/detection_loss‚2.3688810834517846 _step‚464 /loss/FBt1/detection_loss‚2.2254512872960834 0loss/FRt2/assignment_loss‚1.2587752622716568 1loss/SRqqt3/assignment_loss‚1.795442331996229 2loss/SRqqt4/assignment_loss‚1.7564246060756536 1loss/FBt1/assignment_loss‚0.20238365929809435 0loss/FBt4/assignment_loss‚0.1257312060759152 /loss/FRt2/detection_loss‚1.6211636028453416 1loss/SRqqt2/detection_loss‚1.0033282290355567 1ÛMJ‚loss/FBt2/assignment_loss‚0.11613539160787578 /loss/FRt4/detection_loss‚2.6046166436539755 1loss/SRqqt4/detection_loss‚2.3091300693842083 1loss/SRqqt3/detection_loss‚1.7929011750401873 0loss/FBt3/assignment_loss‚0.2777333451177631 .loss/FRt3/detection_loss‚2.621906871256763 trainer/global_step‚5699 _runtime‚5255.802302731 /loss/FRt4/assignment_loss‚3.491817625032531 2loss/SRqqt2/assignment_loss‚1.0041260093817637 /loss/FRt1/detection_loss‚2.4028249132232022 2loss/SRqqt1/assignment_loss‚1.3685261112001568 epoch‚9 1loss/SRqqt1/detection_loss‚1.9063142039398155 $loss/total_loss‚41.1251359380601 % _timestamp‚1.7883945000465178e+09 0loss/FRt1/assignment_loss‚3.3896545515610623 0loss/FRt3/assignment_loss‚2.8717988027285224 /loss/FBt2/detection_loss‚0.4519248605753422 /loss/FBt3/detection_loss‚2.1581440960106097L|U—b­Ü /loss/FRt4/assignment_loss‚3.491817625032531 2loss/SRqqt2/assignment_loss‚1.0041260093817637 /loss/FRt1/detection_loss‚2.4028249132232022 2loss/SRqqt1/assignment_loss‚1.3685261112001568 epoch‚9 1loss/SRqqt1/detection_loss‚1.9063142039398155 $loss/total_loss‚41.1251359380601 % _timestamp‚1.7883945000465178e+09 0loss/FRt1/assignment_loss‚3.3896545515610623 0loss/FRt3/assignment_loss‚2.8717988027285224 /loss/FBt2/detection_loss‚0.4519248605753422 /loss/FBt3/detection_loss‚2.1581440960106097 /loss/FBt4/detection_loss‚2.3688810834517846 _step‚464 /loss/FBt1/detection_loss‚2.2254512872960834 0loss/FRt2/assignment_loss‚1.2587752622716568 1loss/SRqqt3/assignment_loss‚1.795442331996229 2loss/SRqqt4/assignment_loss‚1.7564246060756536 1loss/FBt1/assignment_loss‚0.20238365929809435 0loss/FBt4/assignment_loss‚0.1257312060759152 /loss/FRt2/detection_loss‚1.6211636028453416 1loss/SRqqt2/detection_loss‚1.0033282290355567 1loss/FBt2/assignment_loss‚0.11613539160787578 /loss/FRt4/detection_loss‚2.6046166436539755 1loss/SRqqt4/detection_loss‚2.3091300693842083 1loss/SRqqt3/detection_loss‚1.7929011750401873 0loss/FBt3/assignment_loss‚0.2777333451177631 .loss/FRt3/detection_loss‚2.621906871256763 trainer/global_step‚5699 _runtime‚5255.802302731Ъ¡Û’®‚(:‡ øâÔùé±u gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.368' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.2736 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0»Š^ƒS¯‚(:È øâÔÅ—´x disk.sda3.out‚ 1806.45703125 proc.cpu.threads‚17 disk.drbd1777.out‚0 disk.sda2.out‚0 disk.rbd3.in‚0 disk.dm-1.out‚ 1806.45703125 disk.rbd1.out‚ 2.822265625 disk.rbd0.out‚ 0.2978515625# disk.dm-0.out‚52139.06689453125! disk.rbd2.out‚437.42041015625 disk.rbd0.in‚0$ memory_percent‚93.01528930664062# proc.memory.rssMB‚ 2894.58203125% disk./.usageGB‚1065.8914909362793 disk.sda2.in‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.6328125& disk.nvme0n1.out‚52139.06689453125 cpu‚25.70216441868155) disk./.usagePercent‚71.53287229926389( disk.nvme0c0n1.out‚52139.06689453125 disk.dm-0.in‚ 265.6328125$ proc.memory.availableMB‚1144.375* proc.memory.percent‚17.667126655578613 network.recv‚ 1.807956e+06 disk.drbd1777.in‚0 disk.rbd3.out‚89.89892578125 disk.dm-1.in‚0.125 network.sent‚ 1.2018417e+07 disk.sda3.in‚0.125 disk.sda1.out‚0 disk.sda1.in‚0 disk.rbd1.in‚0 disk.sda.in‚0.125 disk.sda.out‚ 1806.45703125! disk.nvme0c0n1.in‚ 265.6328125rgIÚ’°‚(:‡ œøâÔŸ°u gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.144' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.4288 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0î„œI±‚(:¾ œøâÔ¡·öw network.recv‚ 1.813293e+06 disk.nvme0n1.in‚ 265.6328125 disk.sda.in‚0.125 disk.sda.out‚1823.875 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.sda1.out‚0( disk.nvme0c0n1.out‚52139.68896484375 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd3.in‚0 disk.sda1.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125 network.sent‚ 1.2105092e+07% disk./.usageGB‚1065.8915100097656 disk.dm-1.in‚0.125 disk.rbd1.in‚0 proc.cpu.threads‚17 disk.rbd0.out‚ 0.2978515625! disk.rbd2.out‚438.93212890625& disk.nvme0n1.out‚52139.68896484375( proc.memory.availableMB‚ 1144.1328125) disk./.usagePercent‚71.53287357930155 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 265.6328125# proc.memory.rssMB‚ 2894.58203125* proc.memory.percent‚17.667126655578613 cpu‚25.702023875240613 disk.rbd2.in‚0 disk.rbd3.out‚89.89892578125 disk.dm-1.out‚1823.875 disk.rbd1.out‚ 2.822265625$ memory_percent‚93.01676750183105# disk.dm-0.out‚52139.68896484375 disk.sda3.out‚1823.875™Ù9Ͳ‚(:’ «øâÔÖ®µu gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚149.848' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.969600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ûÏhQ³‚(:Æ «øâÔñ¯×w disk.dm-1.in‚0.125 disk.dm-1.out‚ 1825.359375 disk.drbd1777.out‚0# disk.dm-0.out‚52238.06787109375 network.sent‚ 1.2111656e+07 disk.dm-0.in‚ 265.6328125) disk./.usagePercent‚71.53287511534676 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda2.in‚0 disk.rbd3.out‚89.89892578125 disk.rbd3.in‚0 disk.rbd1.out‚ 2.822265625! disk.rbd2.out‚439.10791015625 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda3.out‚ 1825.359375 proc.cpu.threads‚17 disk.rbd1.in‚0& disk.nvme0n1.out‚52238.06787109375 disk.rbd0.in‚0 disk.drbd1777.in‚0 disk.sda3.in‚0.125 disk.sda.out‚ 1825.359375 disk.sda2.out‚0) proc.memory.availableMB‚ 1100.19140625* proc.memory.percent‚17.667126655578613 cpu‚25.701835456395624 network.recv‚ 1.81824e+06( disk.nvme0c0n1.out‚52238.06787109375# memory_percent‚93.2849645614624# proc.memory.rssMB‚ 2894.58203125% disk./.usageGB‚1065.8915328979492 disk.sda.in‚0.125 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 265.63281254²N®2´- _wandbruntime‚5297  _runtime‚5297ò­%ɵà % _timestamp‚1.7883945419789736e+09 _runtime‚5297.734948229 _step‚465 ' lr-AdamW/pg1‚1.2123155345035045e-05 ' lr-AdamW/pg2‚1.2123155345035045e-05 trainer/global_step‚5749ˆÅáÎ¶È % _timestamp‚1.7883945419789736e+09 _runtime‚5297.734948229 _step‚465 ' lr-AdamW/pg1‚1.2123155345035045e-05 ' lr-AdamW/pg2‚1.2123155345035045e-05 trainer/global_step‚5749Ñ@Å×ä2·- _wandbruntime‚5297  _runtime‚5297›9 ì'¸æO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚98553982 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚386887048588288 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚389994170306048 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚466 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 122606557 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚391625196155392 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4737951388160 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚386887031548928 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30876450 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚377505944766464 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4738147567104 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚98553984 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚394972609326592 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚98553984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30876962 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4711526243522 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚91261338 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 122606557 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚382217680045886 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚386887048588288 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4737951388160 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚98553982 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 122629155 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚391624982937088 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚394972872161792 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 trainer/global_step‚5749 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚391624982937088 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚60384891 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚391625196155392 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚91261853 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 221183139 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚60384888 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0,Lñ'¹ëO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚98553984 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚394972609326592 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚98553984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚515 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚30876962 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4711526243522 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚91261338 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 122606557 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚382217680045886 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚386887048588288 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4737951388160 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚98553982 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 122629155 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚391624982937088 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚394972872161792 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 237931520 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 trainer/global_step‚5749 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚391624982937088 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚60384891 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚391625196155392 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚91261853 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 221183139 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚60384888 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚98553982 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚386887048588288 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚389994170306048 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _step‚466 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 122606557 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚391625196155392 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4737951388160 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 262835200 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚386887031548928 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚30876450 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚377505944766464 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4738147567104 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243Òr\»v2º- _wandbruntime‚5298  _runtime‚529839Xô»ïM >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚30881625 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4738749778944 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚377569561190400 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 221197931 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚467 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚98570504 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚386952070585856 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚91276547 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚5749 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4738945971200 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4712320126470 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚98570504 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚98570502 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚390059695802368 _runtime‚5298.512354457 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚395038976116224 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 221220557 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 221220557 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4712512123218 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4738749778944 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚386952087625216 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 221197931 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚391691033596416 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 122650053 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4979280313856 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0âÕ‘ú&¼ôM EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 221197931 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 _step‚467 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚98570504 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚386952070585856 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚91276547 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚5749 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4738945971200 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4712320126470 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚98570504 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚98570502 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚390059695802368 _runtime‚5298.512354457 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚523 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚395038976116224 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 235821056 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 221220557 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 221220557 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4712512123218 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4738749778944 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚386952087625216 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 221197931 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚391691033596416 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 122650053 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4979280313856 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚30881625 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4738749778944 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚377569561190400 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750Ó‰Ñl2½- _wandbruntime‚5298  _runtime‚5298^‘j(X¾Ò /loss/FRt3/assignment_loss‚3.102463732895098 2loss/SRqqt1/assignment_loss‚1.4047546478142159 1loss/FBt2/assignment_loss‚0.12807510709821837 1loss/FBt3/assignment_loss‚0.22806732005015615 .loss/FRt3/detection_loss‚2.416634979687239 1loss/SRqqt1/detection_loss‚1.7638877725929296 .loss/FBt1/detection_loss‚2.140319774025365 0loss/FBt1/assignment_loss‚0.2319752967713564 /loss/FRt4/detection_loss‚2.8116394629922965 _runtime‚5298.513475851 2loss/SRqqt2/assignment_loss‚0.9166557396864136 0loss/FRt2/assignment_loss‚1.2636265035937815 -loss/FBt3/detection_loss‚1.84479839745022 trainer/global_step‚5749 /loss/FBt2/detection_loss‚0.5937039220716986 % _timestamp‚1.7883945427576654e+09 epoch‚9 1loss/SRqqt4/assignment_loss‚1.750058921513288 /loss/FRt1/detection_loss‚2.2913905508377974 .loss/FRt2/detection_loss‚1.505279395765826 0loss/SRqqt3/detection_loss‚1.974655012478647 0loss/SRqqt4/detection_loss‚2.255925718235643 0loss/FRt1/assignment_loss‚2.7069476366043093 2loss/SRqqt3/assignment_loss‚1.3100677193344936 1loss/FBt4/assignment_loss‚0.10152691348312973 0loss/SRqqt2/detection_loss‚0.875505269390263 %loss/total_loss‚40.67651518194134 _step‚468 0loss/FRt4/assignment_loss‚3.6161509873503346 /loss/FBt4/detection_loss‚3.4424044002186167‰D€]¿× .loss/FBt1/detection_loss‚2.140319774025365 0loss/FBt1/assignment_loss‚0.2319752967713564 /loss/FRt4/detection_loss‚2.8116394629922965 _runtime‚5298.513475851 2loss/SRqqt2/assignment_loss‚0.9166557396864136 0loss/FRt2/assignment_loss‚1.2636265035937815 -loss/FBt3/detection_loss‚1.84479839745022 trainer/global_step‚5749 /loss/FBt2/detection_loss‚0.5937039220716986 % _timestamp‚1.7883945427576654e+09 epoch‚9 1loss/SRqqt4/assignment_loss‚1.750058921513288 /loss/FRt1/detection_loss‚2.2913905508377974 .loss/FRt2/detection_loss‚1.505279395765826 0loss/SRqqt3/detection_loss‚1.974655012478647 0loss/SRqqt4/detection_loss‚2.255925718235643 0loss/FRt1/assignment_loss‚2.7069476366043093 2loss/SRqqt3/assignment_loss‚1.3100677193344936 1loss/FBt4/assignment_loss‚0.10152691348312973 0loss/SRqqt2/detection_loss‚0.875505269390263 %loss/total_loss‚40.67651518194134 _step‚468 0loss/FRt4/assignment_loss‚3.6161509873503346 /loss/FBt4/detection_loss‚3.4424044002186167 /loss/FRt3/assignment_loss‚3.102463732895098 2loss/SRqqt1/assignment_loss‚1.4047546478142159 1loss/FBt2/assignment_loss‚0.12807510709821837 1loss/FBt3/assignment_loss‚0.22806732005015615 .loss/FRt3/detection_loss‚2.416634979687239 1loss/SRqqt1/detection_loss‚1.7638877725929296Ôç9AýÀ‚(:… ºøâÔ‘¡ºu gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.26' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.452 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0z¹ÉƒFÁ‚(:» ºøâÔïÿ†x! disk.rbd2.out‚440.61962890625 disk.sda.in‚0.125 disk.sda.out‚ 1842.6796875 disk.dm-1.out‚ 1842.6796875 disk.rbd1.in‚0 disk.rbd1.out‚ 2.822265625& proc.memory.availableMB‚ 1015.96875# proc.memory.rssMB‚ 2894.58203125 disk.rbd0.in‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125$ memory_percent‚93.79901885986328 proc.cpu.threads‚17$ disk.nvme0c0n1.out‚ 52336.6796875 disk.sda2.in‚0) disk./.usagePercent‚71.53287665139196 disk.rbd0.out‚ 0.2978515625! disk.nvme0c0n1.in‚ 265.6328125 disk.dm-0.out‚ 52336.6796875* proc.memory.percent‚17.667126655578613" disk.nvme0n1.out‚ 52336.6796875 disk.dm-1.in‚0.125 disk.nvme0n1.in‚ 265.6328125 cpu‚25.70178051173745 disk.sda1.in‚0 disk.sda3.in‚0.125 disk.sda3.out‚ 1842.6796875 disk.rbd2.in‚0 network.sent‚ 1.2117081e+07 network.recv‚ 1.821771e+06 disk.sda1.out‚0 disk.sda2.out‚0% disk./.usageGB‚1065.8915557861328 disk.drbd1777.in‚0 disk.drbd1777.out‚0 disk.rbd3.out‚89.89892578125©è®ö’‚(:‡ ÉøâÔ–æ¶u gpu.0.gpu‚21.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.672' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.1344 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'{H6XÂ(:Í ÉøâÔéÇ’y disk.dm-0.in‚ 265.6328125$ memory_percent‚93.47946643829346* proc.memory.percent‚17.667126655578613( disk.nvme0c0n1.out‚52337.07958984375 disk.rbd3.in‚0 disk.rbd3.out‚89.89892578125& disk.nvme0n1.out‚52337.07958984375% disk./.usageGB‚1065.8916282653809! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.in‚0.125# proc.memory.rssMB‚ 2894.58203125 disk.drbd1777.out‚0 cpu‚25.701626843772512) disk./.usagePercent‚71.53288151553511 disk.sda2.in‚0# disk.dm-0.out‚52337.07958984375 disk.rbd1.out‚ 2.822265625 disk.sda.out‚ 1856.23046875 disk.sda1.in‚0 disk.drbd1777.in‚0) proc.memory.availableMB‚ 1068.32421875 network.recv‚ 1.827092e+06 disk.sda3.out‚ 1856.23046875 disk.dm-1.out‚ 1856.23046875 disk.rbd1.in‚0 disk.rbd2.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda.in‚0.125 disk.sda1.out‚0 disk.dm-1.in‚0.125 disk.rbd2.out‚442.3193359375 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 proc.cpu.threads‚17 network.sent‚ 1.2204093e+07 disk.sda2.out‚0¨„Wâ’Ä‚(:‡ ØøâÔÀ¬Ûu gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚103.317' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.6634 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý<ÆgRÅ‚(:Ç ØøâÔþ˜™x disk.sda3.in‚0.125 disk.sda.in‚0.125 disk.drbd1777.out‚0 network.sent‚ 1.2210208e+07 disk.dm-1.in‚0.125 disk.sda1.out‚0 disk.rbd3.in‚0$ memory_percent‚93.89212131500244* proc.memory.percent‚17.667126655578613 cpu‚25.701401929325208% disk./.usageGB‚1065.8878860473633 disk.rbd2.in‚0 disk.sda.out‚ 1867.984375 disk.rbd0.in‚0) proc.memory.availableMB‚ 1000.71484375# proc.memory.rssMB‚ 2894.58203125 proc.cpu.threads‚17 disk.rbd0.out‚ 0.2978515625 disk.drbd1777.in‚0 disk.sda2.in‚0 disk.nvme0n1.in‚ 265.6328125 disk.sda2.out‚0 disk.rbd3.out‚89.89892578125 disk.dm-1.out‚ 1867.984375 disk.rbd1.out‚ 2.822265625! disk.nvme0c0n1.in‚ 265.6328125) disk./.usagePercent‚71.53263037214403& disk.nvme0n1.out‚52435.81298828125( disk.nvme0c0n1.out‚52435.81298828125 disk.dm-0.in‚ 265.6328125# disk.dm-0.out‚52435.81298828125 network.recv‚ 1.831502e+06 disk.sda3.out‚ 1867.984375 disk.rbd1.in‚0 disk.rbd2.out‚442.3193359375 disk.sda1.in‚0{>çb2Æ- _wandbruntime‚5340  _runtime‚5340ªß«¯ÇÇÁ & lr-AdamW/pg1‚8.321189994872429e-06 & lr-AdamW/pg2‚8.321189994872429e-06 trainer/global_step‚5799 % _timestamp‚1.7883945845282207e+09 _runtime‚5340.284274189 _step‚469›¡YÌÈÆ & lr-AdamW/pg1‚8.321189994872429e-06 & lr-AdamW/pg2‚8.321189994872429e-06 trainer/global_step‚5799 % _timestamp‚1.7883945845282207e+09 _runtime‚5340.284274189 _step‚469Õž’jÙ2É- _wandbruntime‚5340  _runtime‚5340ÒØ­4&ÊæO VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚394917095369216 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4778094934528 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4751439559470 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚390139000434688 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 _runtime‚5340.285535666 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4778094934528 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚92022851 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚60886588 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚99379982 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252611584 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 123676107 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 223056091 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚92023348 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚99379984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚497 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚398290812512768 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚385437370594094 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚390139000434688 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4751247562930 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚394917095369216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚380685931034624 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚390138983395328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚398291090028032 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5020620002304 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5020367390720 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚394916882150912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚393270470025728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31136263 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 223033491 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚394916882150912 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4777898755584 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 123653509 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚470 % _timestamp‚1.7883945845293329e+09 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31136757 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚380685913995264 trainer/global_step‚5799 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 123676107 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 277515264 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚390138983395328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚99379984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚385437161558194 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚393270445122048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0l÷X—Ù @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 123653509 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚99379982]½ñ'ËëO GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5020367390720 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚394916882150912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚393270470025728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31136263 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 223033491 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚394916882150912 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4777898755584 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 123653509 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚470 % _timestamp‚1.7883945845293329e+09 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31136757 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚380685913995264 trainer/global_step‚5799 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 123676107 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 277515264 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚390138983395328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚99379984 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚385437161558194 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚393270445122048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 123653509 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚99379982 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚394917095369216 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4778094934528 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4751439559470 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚390139000434688 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 _runtime‚5340.285535666 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4778094934528 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚92022851 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚60886588 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚99379982 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 252611584 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 123676107 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 223056091 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚92023348 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚99379984 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚497 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚398290812512768 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚385437370594094 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚390139000434688 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4751247562930 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚394917095369216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚380685931034624 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚390138983395328 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚398291090028032 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5020620002304ÖÒ¡2Ì- _wandbruntime‚5341  _runtime‚5341â¿ö&ÍðM H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 223093509 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5021189953024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚60896622 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4778697146368 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 223093509 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5021444648448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚92038418 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚385501554825718 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4778893338624 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4778697146368 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚99396502 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚398357160571392 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 223070883 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 123674381 % _timestamp‚1.7883945853075566e+09 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚99396504 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚390204039471616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚394982719578624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚398357440170496 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚390204022432256 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚380749530419200 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚99396502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚393335995522048 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4752041445878 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚471 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚522 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚31141271 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚380749513379840 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4778893338624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚92037893 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚394982932810240 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 123674381 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚385501763861826 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚00H Ëû&ÎõM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚390204039471616 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚390204022432256 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4752233442626 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 123697005 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚390204022432256 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚380749530419200 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚99396502 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚393335995522048 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4752041445878 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 _step‚471 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 10869712896 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚522 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚31141271 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚380749513379840 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4778893338624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚92037893 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚394982932810240 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 123674381 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚385501763861826 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 223093509 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5021189953024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚60896622 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4778697146368 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 223093509 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5021444648448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚92038418 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚385501554825718 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4778893338624 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4778697146368 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚99396502 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚398357160571392 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 223070883 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 123674381 % _timestamp‚1.7883945853075566e+09 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚99396504 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚390204039471616 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚394982719578624 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚398357440170496 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 Fî,ÈØ $ lr-AdamW/pg1‚5.2297297958136e-06 $ lr-AdamW/pg2‚5.2297297958136e-06 trainer/global_step‚5849 % _timestamp‚1.7883946268777506e+09 _runtime‚5382.633719497 _step‚473Ù‚«þ2Ù- _wandbruntime‚5382  _runtime‚5382)xÈ:ð'ÚêO XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚ 100205982 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 124723059 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4817846123008 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚393390935241728 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚398208781364736 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 224906443 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 100205984 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4817846123008 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚61388288 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚398208781364736 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4818042301952 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚388656852106402 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚401609264299520 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚388657061142302 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚61388291 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚396546719938048 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 % _timestamp‚1.7883946268788776e+09 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31396238 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4791160878878 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4818042301952 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚519 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 124700461 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚522 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚398208994583040 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 124723059 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚92784007 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 _step‚474 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5062519457792 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚398208994583040 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 124700461 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 trainer/global_step‚5849 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 246320128 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚393390952281088 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5062273137664 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚393390952281088 _runtime‚5382.635125056 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚383865900263424 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚92784529 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚393390952281088 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5062273137664 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚393390952281088 _runtime‚5382.635125056 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚383865900263424 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚92784529 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4817846123008 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚393390935241728 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚398208781364736 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 224906443 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 100205984 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4817846123008 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚61388288 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚398208781364736 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4818042301952 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚388656852106402 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚401609264299520 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚388657061142302 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚61388291 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 271223808 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560ÚΉ`32Ü- _wandbruntime‚5383  _runtime‚5383™q°´ÝôM JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚539 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 224966461 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 124743957 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚388721454410034 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5063354030592 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 _step‚475 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚398274618792448 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚388721245373926 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚383929499648000 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚92799257 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚398274832024064 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 100222502 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 100222504 OADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚ 100222502 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 224966461 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚383929482608640 SEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚ 100222504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚393455974278656 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚92799799 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4791762765286 JBDeviceStatsMonitor.on_train_baZ•Ftch_end/segment.large_pool.allocated‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4818644513792 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4791954762034 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚396612270338048 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚398274832024064 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 224966461 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚ 124743957 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 244209664 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚388721454410034 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5063354030592 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 _step‚475 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚398274618792448 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3065 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚388721245373926 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚383929499648000 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚92799257 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚398274832024064 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 100222502 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 100222504 OADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚ 100222502 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 269113344 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 224966461 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚383929482608640 SEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚ 100222504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚393455974278656 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚92799799 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4791762765286 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4818644513792 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4791954762034 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚396612270338048 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚243 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚398274832024064 Eï'ìéO \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚396642887088128 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _step‚478 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚396642904127488 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4857793490432 QCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚ 101031982 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚479 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 101031984 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5104260480000 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚391876542654610 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚401500680578560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚387045869492224 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚396642904127488 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚396642887088128 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 125747413 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚401500893796864 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚401500680578560 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31655147 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 _runtime‚5425.330984751 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚404927255234048 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4830882198286 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4830690201746 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚ 101031982 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 $ _timestamp‚1.788394669574807e+09 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚93544659 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚ 101031984 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚391876751690510 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31654671 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4857793490432 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚404927522263552 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚399822994754048 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚5899 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4857989669376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚93545138 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 125747413 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 125770011 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 226779395 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065|aöË íîO JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M?DeviceStatsMonitor.on_train_batch_start/activec*²‡).large_pool.freed‚ 101031982 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 $ _timestamp‚1.788394669574807e+09 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚93544659 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 196178944 UGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚ 101031984 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚391876751690510 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 213218304 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31654671 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 242125824 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4857793490432 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚404927522263552 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚399822994754048 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚5899 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4857989669376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚93545138 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 125747413 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 125770011 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 226779395 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚396642887088128 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _step‚478 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚396642904127488 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4857793490432 QCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚ 101031982 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚479 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 101031984 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5104260480000 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚391876542654610 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚401500680578560 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚387045869492224 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚396642904127488 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚396642887088128 M?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚ 125747413 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚401500893796864 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚401500680578560 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31655147 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 _runtime‚5425.330984751 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 267029504 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚404927255234048 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4830882198286 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4830690201746 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507Þ»_¿„2î- _wandbruntime‚5426  _runtime‚5426¯w^ú&ïôM KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 509607936 trainer/global_step‚5899 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4858591881216 OADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚ 101048502 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 226816787 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚5426.098436042 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 226839413 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 226839413 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 125768285 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 101048502 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚396707926125056 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4858788073472 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4858788073472 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚404993885349376 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 101048504 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4831484084694 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚396707943164416 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚31659713 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚387109468876800 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚401566731237888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5105340195328 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 125790909 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚61900025 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚61900022 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚31660244 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚391941144958242 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚399888545154048 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚391940935922134 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚534 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883946703419135e+09 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 226816787 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚396707926125056 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚93559735 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚507 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 101048504 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4831484084694 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 11559501824 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚396707943164416 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚31659713 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚387109468876800 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚401566731237888 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5105340195328 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚27581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1148 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 125790909 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚61900025 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚61900022 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚31660244 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚391941144958242 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚399888545154048 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 254695424 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚391940935922134 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚534 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 279599104 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 % _timestamp‚1.7883946703419135e+09 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 226816787 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚396707926125056 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚93559735 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 226816787 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚5426.098436042 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 226839413 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 226839413 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 125768285 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 101048502 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚396707926125056 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4858788073472 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4858788073472 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚404993885349376 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 Pl22ý- _wandbruntime‚5467  _runtime‚5467aöô#ï'þéO P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚404792579792384 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚ 101857984 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚403099269570048 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚395096442238718 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚390225821681664 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788394711758505e+09 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚ 101857982 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5146499151360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4897740857856 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚482 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚404792793010688 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 126794365 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 126816963 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4870411521154 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 101857984 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4897740857856 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚399894855973888 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚399894838934528 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚408245793625088 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚395096233202818 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚94304199 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4897937036800 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31913021 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31912511 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚513 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚403099294473728 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5146259122688 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚399894838934528 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4897937036800 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4870603517694 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 M?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚ 101857982 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚62391691 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚408245528692736 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚399894838934528 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4897937036800 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4870603517694 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 196178944 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 M?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚ 101857982 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚62391691 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚408245528692736 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 10958640713 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚404792579792384 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 10869712896 UGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚ 101857984 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚403099269570048 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 11559501824 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1148 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚395096442238718 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 209035900 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 357474816 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚750 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 264932352 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚390225821681664 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 494345553 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 $ _timestamp‚1.788394711758505e+09 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 QCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚ 101857982 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5146499151360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 11322690560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 10869712896 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4897740857856 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚482 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 498534912 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚404792793010688 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 126794365 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 11049893888 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 UGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚ 126816963 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4870411521154 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 QCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚ 101857984 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4897740857856 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚243 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 11559501824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚399894855973888 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚399894838934528 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚22598 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚408245793625088 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚395096233202818 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3065 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚94304199 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 240028672 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚507 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4897937036800 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚31913021 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚31912511 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚27328 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚513 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚403099294473728 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5146259122688 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚62401722 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚403164819970048 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚483 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 101874504 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚404858630451712 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4898539248640 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 228689739 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 228712365 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚399959895010816 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚530 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚408312168572928 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 254433280 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚62401725 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚395160626470342 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚94319319 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 trainer/global_step‚5949 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5147119073280 _runtime‚5468.260719062 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4898735440896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚395160626470342 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚94319319 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 156469760 trainer/global_step‚5949 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5147119073280 _runtime‚5468.260719062 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4898735440896 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚62401722 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚403164819970048 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚483 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 101874504 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚404858630451712 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4898539248640 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 228689739 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 228712365 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚399959895010816 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0ã¶%õª2ƒ- _wandbruntime‚5468  _runtime‚5468BÆH7X„Ò 2loss/SRqqt3/assignment_loss‚1.4642216849248284 .loss/FRt1/detection_loss‚2.391067446294156 .loss/FRt4/detection_loss‚2.773503548465669 0loss/SRqqt3/detection_loss‚1.718386084557726 $ _timestamp‚1.788394712506081e+09 1loss/FBt1/assignment_loss‚0.39419111276928515 1loss/SRqqt1/detection_loss‚1.9924579597216971 1loss/SRqqt4/detection_loss‚2.2758036761180214 /loss/FBt4/detection_loss‚2.5598889109161167 _runtime‚ 5468.26189056 2loss/SRqqt4/assignment_loss‚1.7041867195588092 /loss/FRt2/detection_loss‚1.8278382197841183 1loss/SRqqt2/detection_loss‚1.0560907441511995 0loss/FRt1/assignment_loss‚3.0040690993720833 /loss/FRt4/assignment_loss‚3.212775418534875 2loss/SRqqt1/assignment_loss‚1.6115718815336995 1loss/FBt4/assignment_loss‚0.22628349228438716 /loss/FBt2/detection_loss‚0.6020288344670358 %loss/total_loss‚41.02782553268348 _step‚484 0loss/FRt2/assignment_loss‚1.2203388561259259 .loss/FRt3/detection_loss‚2.316836388707161 .loss/FRt3/assignment_loss‚3.13940844476223 1loss/FBt3/assignment_loss‚0.21504660842765588 .loss/FBt3/detection_loss‚2.046612061560154 epoch‚9 2loss/SRqqt2/assignment_loss‚0.9254642960333483 1loss/FBt2/assignment_loss‚0.14411657294128818 .loss/FBt1/detection_loss‚2.205637470672005 trainer/global_step‚5949q-Ï]…× 1loss/SRqqt1/detection_loss‚1.9924579597216971 1loss/SRqqt4/detection_loss‚2.2758036761180214 /loss/FBt4/detection_loss‚2.5598889109161167 _runtime‚ 5468.26189056 2loss/SRqqt4/assignment_loss‚1.7041867195588092 /loss/FRt2/detection_loss‚1.8278382197841183 1loss/SRqqt2/detection_loss‚1.0560907441511995 0loss/FRt1/assignment_loss‚3.0040690993720833 /loss/FRt4/assignment_loss‚3.212775418534875 2loss/SRqqt1/assignment_loss‚1.6115718815336995 1loss/FBt4/assignment_loss‚0.22628349228438716 /loss/FBt2/detection_loss‚0.6020288344670358 %loss/total_loss‚41.02782553268348 _step‚484 0loss/FRt2/assignment_loss‚1.2203388561259259 .loss/FRt3/detection_loss‚2.316836388707161 .loss/FRt3/assignment_loss‚3.13940844476223 1loss/FBt3/assignment_loss‚0.21504660842765588 .loss/FBt3/detection_loss‚2.046612061560154 epoch‚9 2loss/SRqqt2/assignment_loss‚0.9254642960333483 1loss/FBt2/assignment_loss‚0.14411657294128818 .loss/FBt1/detection_loss‚2.205637470672005 trainer/global_step‚5949 2loss/SRqqt3/assignment_loss‚1.4642216849248284 .loss/FRt1/detection_loss‚2.391067446294156 .loss/FRt4/detection_loss‚2.773503548465669 0loss/SRqqt3/detection_loss‚1.718386084557726 $ _timestamp‚1.788394712506081e+09 1loss/FBt1/assignment_loss‚0.39419111276928515äz~³ñ’†‚(:‡ ßùâԳäu gpu.0.gpu‚59.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚147.144' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.4288 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0cOÅ$O‡‚(:Ä ßùâÔ«þôw$ disk.nvme0n1.out‚53051.431640625 disk.sda.out‚ 1964.5234375 disk.sda1.out‚0 disk.dm-1.out‚ 1964.5234375 network.sent‚ 1.2507854e+07 network.recv‚ 1.875035e+06! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.out‚ 1964.5234375 disk.rbd1.in‚0 cpu‚25.700250580825724% disk./.usageGB‚1057.2350387573242 disk.sda3.in‚0.125 disk.dm-1.in‚0.125 disk.rbd2.out‚450.6591796875 disk.drbd1777.out‚0 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0! disk.dm-0.out‚53051.431640625 disk.rbd1.out‚ 2.822265625# proc.memory.rssMB‚ 2894.58203125 disk.sda.in‚0.125& disk.nvme0c0n1.out‚53051.431640625) proc.memory.availableMB‚ 1001.44921875 proc.cpu.threads‚17) disk./.usagePercent‚70.95193053028704 disk.rbd3.in‚0* proc.memory.percent‚17.667126655578613 disk.rbd3.out‚89.89892578125 disk.dm-0.in‚ 265.6328125 disk.rbd2.in‚0$ memory_percent‚93.88763904571533 disk.nvme0n1.in‚ 265.6328125 disk.drbd1777.in‚0 disk.sda2.out‚0v=‚(:† îùâÔ’ÌÈu gpu.0.gpu‚59.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚91.477' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.2954 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ˆW5•<‰‚(:± îùâÔÅ€y) disk./.usagePercent‚70.95192899424184 disk.rbd2.out‚452.1708984375 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0 disk.drbd1777.in‚0 disk.sda2.out‚0* proc.memory.percent‚17.667126655578613 network.recv‚ 1.880452e+06% disk./.usageGB‚1057.2350158691406 disk.rbd1.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125( proc.memory.availableMB‚ 1001.4140625 disk.sda1.in‚0 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.dm-1.out‚1979.5 network.sent‚ 1.2594603e+07 disk.sda.in‚0.125& disk.nvme0c0n1.out‚53088.302734375! disk.dm-0.out‚53088.302734375 disk.sda3.out‚1979.5 disk.dm-1.in‚0.125 proc.cpu.threads‚17 disk.rbd1.out‚ 2.822265625 disk.rbd2.in‚0$ disk.nvme0n1.out‚53088.302734375 disk.rbd3.in‚0# proc.memory.rssMB‚ 2894.58203125 disk.rbd0.in‚0$ memory_percent‚93.88785362243652 disk.sda.out‚1979.5 disk.rbd3.out‚89.89892578125 cpu‚25.700239568362154 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0z½<Š‚(:’ ýùâÔ’ÇÁu gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚160.096' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.019200000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0JÌ]ÒP‹‚(:Å ýùâÔ»‹éw disk.dm-0.in‚ 265.6328125 disk.sda3.in‚0.125 disk.dm-1.in‚0.125 disk.rbd1.out‚ 2.822265625 disk.sda.in‚0.125! disk.nvme0c0n1.in‚ 265.6328125) disk./.usagePercent‚70.95192669017403 disk.sda2.out‚0 disk.rbd3.in‚0 disk.rbd2.out‚453.7880859375 proc.cpu.threads‚17% disk./.usageGB‚1057.2349815368652$ disk.nvme0n1.out‚53186.572265625 disk.drbd1777.out‚0$ memory_percent‚93.88785362243652 network.recv‚ 1.884214e+06 disk.rbd3.out‚89.89892578125 disk.sda3.out‚ 1980.72265625* proc.memory.percent‚17.667126655578613 disk.rbd0.out‚ 0.2978515625 network.sent‚ 1.2600167e+07 disk.sda2.in‚0 disk.dm-1.out‚ 1980.72265625 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0# proc.memory.rssMB‚ 2894.58203125 cpu‚25.70011644910435 disk.rbd1.in‚0 disk.sda.out‚ 1980.72265625 disk.sda1.in‚0! disk.dm-0.out‚53186.572265625 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0& disk.nvme0c0n1.out‚53186.572265625( proc.memory.availableMB‚ 1001.4140625ä%õ§2Œ- _wandbruntime‚5508  _runtime‚5508i+ÕŸÆÀ _step‚485 & lr-AdamW/pg1‚2.412131202565409e-07 & lr-AdamW/pg2‚2.412131202565409e-07 trainer/global_step‚5999 % _timestamp‚1.7883947532369041e+09 _runtime‚ 5508.99291899?„DËŽÅ % _timestamp‚1.7883947532369041e+09 _runtime‚ 5508.99291899 _step‚485 & lr-AdamW/pg1‚2.412131202565409e-07 & lr-AdamW/pg2‚2.412131202565409e-07 trainer/global_step‚5999åRlí2- _wandbruntime‚5508  _runtime‚5508™Sóƒð'êO LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 191996540 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚750 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚62893391 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 230547899 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚95064270 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚406375569289728 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4937688225280 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚411563814070272 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 127863915 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚403146807820288 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 _step‚486 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚408084479006208 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 127841317 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4937688225280 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 _runtime‚5508.994158026 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚95063728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 % _timestamp‚1.7883947532379625e+09 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5188503421440 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚27328 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚62893391 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 230547899 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚542 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 233737216 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚750 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 10509293568 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3065 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚95064270 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚406375569289728 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 213218304 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4937688225280 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 156469760 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚411563814070272 GADevi|Q6{ceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 509607936 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚507 QCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚ 127863915 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚403146807820288 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1130 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 11049893888 _step‚486 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚22600 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚408084479006208 QCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚ 127841317 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 509607936 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4937688225280 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 498534912 _runtime‚5508.994158026 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚27581 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚750 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚507 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 11559501824 JDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚22600 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚95063728 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 277050368 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 11049893888 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 509607936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚243 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚27581 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚22598 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 % _timestamp‚1.7883947532379625e+09 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5188503421440 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚243 JDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚403211846857216 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚393469407334400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚408150529665536 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚403211829817856 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚403211829817856 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4938486616064 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4911118720258 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 _runtime‚5509.742165875 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚95079561 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚62903422 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚403211846857216 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚398380526054658 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚ 102700504 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚406441069882368 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚5999 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚411630450020352 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5189117316096 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 127884813 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚62903425 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 102700504 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4938682808320 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚537 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 230585317 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚95079024 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 230585317 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚32175602 _step‚487 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4938486616064 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 127862189 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5189355234304 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚398380317018550 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 230562691 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚411630187198464 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 127862189 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 102700502 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750'Àtaþ&”øM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚22624 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚32176136 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚403211846857216 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 213231616 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚393469407334400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 191996748 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚22626 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 277050368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚750 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚750 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚408150529665536 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚403211829817856 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚403211829817856 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4938486616064 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4911118720258 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 494345553 _runtime‚5509.742165875 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚95079561 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1130 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 209036108 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚62903422 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚403211846857216 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚398380526054658 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 213231616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 196192256 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 11559501824 SEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚ 102700504 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 11322690560 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚406441069882368 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 10958640713 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 509607936 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 11322690560 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 11049893888 trainer/global_step‚5999 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚411630450020352 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 509607936 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5189117316096 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚243 SEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚ 127884813 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 357474816 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚243 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚62903425 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 11559501824 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚507 OADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚ 102700504 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4938682808320 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚537 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 11049893888 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 11049893888 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚22624 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 230585317 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚95079024 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 10509293568 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 230585317 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚32175602 _step‚487 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚27581 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4938486616064 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 237918208 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚750 K=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚ 127862189 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚27328 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 262821888 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5189355234304 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚398380317018550 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 230562691 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚411630187198464 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚507 OADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚ 127862189 K=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚ 102700502 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 498534912 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 10869712896 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚750 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚408150529665536 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 196192256 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 498534912çD°•2•- _wandbruntime‚5509  _runtime‚5509=%I³Y–Ó epoch‚9 trainer/global_step‚5999 /loss/FRt2/assignment_loss‚1.452832309263093 2loss/SRqqt1/assignment_loss‚1.6542808793885402 0loss/FBt3/assignment_loss‚0.2729810164532597 1loss/SRqqt2/assignment_loss‚1.101265237737447 /loss/FBt4/detection_loss‚2.7867521185141344 1loss/SRqqt3/assignment_loss‚1.546528531704098 1loss/SRqqt1/detection_loss‚1.5695973009117379 0loss/SRqqt2/detection_loss‚1.007466353431344 %loss/total_loss‚41.28104835627641 1loss/FBt2/assignment_loss‚0.17491119047810091 1loss/SRqqt4/detection_loss‚2.1922907985746862 % _timestamp‚1.7883947539874804e+09 /loss/FBt1/assignment_loss‚0.214497877446614 1loss/FBt4/assignment_loss‚0.13702926915721036 1loss/SRqqt3/detection_loss‚1.7508081964084081 .loss/FBt1/detection_loss‚2.167939231676214 .loss/FBt3/detection_loss‚2.491115191403557 _step‚488 0loss/FRt3/assignment_loss‚2.9677443656656477 /loss/FRt4/assignment_loss‚3.413628674390023 /loss/FRt1/detection_loss‚2.3438583244668676 _runtime‚5509.743266429 0loss/FRt1/assignment_loss‚2.9911900796788804 2loss/SRqqt4/assignment_loss‚1.8941572890151293 /loss/FRt3/detection_loss‚2.3549459771977532 /loss/FRt4/detection_loss‚2.6207588288867685 /loss/FBt2/detection_loss‚0.5472921946396431 /loss/FRt2/detection_loss‚1.6271771197872502€EË^—Ø 0loss/FBt3/assignment_loss‚0.2729810164532597 1loss/SRqqt2/assignment_loss‚1.101265237737447 /loss/FBt4/detection_loss‚2.7867521185141344 1loss/SRqqt3/assignment_loss‚1.546528531704098 1loss/SRqqt1/detection_loss‚1.5695973009117379 0loss/SRqqt2/detection_loss‚1.007466353431344 %loss/total_loss‚41.28104835627641 1loss/FBt2/assignment_loss‚0.17491119047810091 1loss/SRqqt4/detection_loss‚2.1922907985746862 % _timestamp‚1.7883947539874804e+09 /loss/FBt1/assignment_loss‚0.214497877446614 1loss/FBt4/assignment_loss‚0.13702926915721036 1loss/SRqqt3/detection_loss‚1.7508081964084081 .loss/FBt1/detection_loss‚2.167939231676214 .loss/FBt3/detection_loss‚2.491115191403557 _step‚488 0loss/FRt3/assignment_loss‚2.9677443656656477 /loss/FRt4/assignment_loss‚3.413628674390023 /loss/FRt1/detection_loss‚2.3438583244668676 _runtime‚5509.743266429 0loss/FRt1/assignment_loss‚2.9911900796788804 2loss/SRqqt4/assignment_loss‚1.8941572890151293 /loss/FRt3/detection_loss‚2.3549459771977532 /loss/FRt4/detection_loss‚2.6207588288867685 /loss/FBt2/detection_loss‚0.5472921946396431 /loss/FRt2/detection_loss‚1.6271771197872502 epoch‚9 trainer/global_step‚5999 /loss/FRt2/assignment_loss‚1.452832309263093 2loss/SRqqt1/assignment_loss‚1.6542808793885402è­Ø=k’˜‚(:‡ ŒúâÔàà¸w gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.842' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.7684 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ô `ËU™‚(:Ê ŒúâÔÏÄày disk.rbd3.out‚89.89892578125 network.sent‚ 1.2606389e+07 disk.sda2.in‚0 disk.rbd1.out‚ 2.822265625 disk.rbd2.in‚0 disk.drbd1777.out‚0 disk.rbd3.in‚0 disk.dm-0.in‚ 265.6328125 disk.rbd2.out‚453.7880859375 disk.sda.out‚ 1981.42578125# proc.memory.rssMB‚ 2894.58203125 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.out‚0 disk.dm-1.in‚0.125 disk.sda1.in‚0 disk.rbd0.out‚ 0.2978515625 disk.sda.in‚0.125 proc.cpu.threads‚17) disk./.usagePercent‚70.95192669017403% disk./.usageGB‚1057.2349815368652( disk.nvme0c0n1.out‚53285.05419921875 network.recv‚ 1.88881e+06 disk.dm-1.out‚ 1981.42578125 disk.rbd1.in‚0& disk.nvme0n1.out‚53285.05419921875 cpu‚25.70000888619943# disk.dm-0.out‚53285.05419921875 disk.nvme0n1.in‚ 265.6328125 disk.sda1.out‚0# memory_percent‚93.8877820968628) proc.memory.availableMB‚ 1001.42578125 disk.sda3.in‚0.125 disk.sda3.out‚ 1981.42578125 disk.rbd0.in‚0* proc.memory.percent‚17.667126655578613æíˆU’š‚(:‡ ›úâÔº«Ûu gpu.0.gpu‚55.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚34.0 gpu.0.powerWatts‚145.791' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.1582 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚05áaV›‚(:Ë ›úâÔ…ôºw disk.rbd2.out‚455.4052734375 disk.sda1.out‚0# disk.dm-0.out‚53285.40966796875* proc.memory.percent‚17.667126655578613 proc.cpu.threads‚17 network.recv‚ 1.894811e+06 disk.rbd2.in‚0& disk.nvme0n1.out‚53285.40966796875 disk.drbd1777.out‚0) disk./.usagePercent‚70.95195741107814$ disk./.usageGB‚1057.235439300537 disk.dm-1.out‚ 1981.87890625 disk.sda.in‚0.125( disk.nvme0c0n1.out‚53285.40966796875 disk.sda3.in‚0.125# memory_percent‚93.8880205154419 disk.sda.out‚ 1981.87890625 disk.rbd0.out‚ 0.2978515625 network.sent‚ 1.2694291e+07 disk.sda1.in‚0 disk.dm-0.in‚ 265.6328125 disk.sda3.out‚ 1981.87890625# proc.memory.rssMB‚ 2894.58203125 disk.dm-1.in‚0.125 disk.nvme0n1.in‚ 265.6328125 disk.rbd0.in‚0 disk.rbd3.in‚0 cpu‚25.699941914176353 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.in‚0) proc.memory.availableMB‚ 1001.38671875 disk.rbd1.in‚0 disk.rbd1.out‚ 2.822265625 disk.sda2.out‚0 disk.rbd3.out‚89.89892578125ø»‘œ‚(:… ªúâÔÛÙu gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚89.405' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚17.881 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0—o¹‚(:½ ªúâÔò²éw( proc.memory.availableMB‚ 981.52734375 proc.cpu.threads‚17 disk.sda2.out‚0" disk.dm-0.out‚53384.6650390625 disk.dm-0.in‚ 265.6328125$ disØjqRk./.usageGB‚1057.235439300537 disk.rbd1.in‚0 disk.drbd1777.in‚0 disk.sda1.out‚0 disk.rbd3.in‚0 cpu‚25.706880150510926 disk.rbd2.out‚455.4052734375 disk.sda1.in‚0 disk.rbd0.in‚0# proc.memory.rssMB‚ 2914.51171875 network.recv‚ 1.898694e+06) disk./.usagePercent‚70.95195741107814 disk.rbd1.out‚ 2.822265625 disk.nvme0n1.in‚ 265.6328125' disk.nvme0c0n1.out‚53384.6650390625 disk.sda3.out‚ 1982.28125 network.sent‚ 1.269963e+07 disk.rbd2.in‚0 disk.sda.in‚0.125 disk.drbd1777.out‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.dm-1.out‚ 1982.28125) proc.memory.percent‚17.78876781463623 disk.sda.out‚ 1982.28125 disk.rbd0.out‚ 0.2978515625 disk.sda2.in‚0 disk.rbd3.out‚89.89892578125 disk.dm-1.in‚0.125% disk.nvme0n1.out‚53384.6650390625 disk.sda3.in‚0.125$ memory_percent‚94.00923252105713+ç4a’ž‚(:‡ ¹úâÔ‰®Ëu gpu.0.gpu‚69.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚32.0 gpu.0.powerWatts‚132.492' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.4984 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0àêîRŸ‚(:Ç ¹úâÔ€˜x) disk./.usagePercent‚70.95195741107814! disk.nvme0c0n1.in‚ 265.6328125 disk.sda2.out‚0 disk.dm-1.out‚ 1982.98046875 disk.nvme0n1.in‚ 265.6328125 disk.drbd1777.out‚0) proc.memory.percent‚17.75214672088623 disk.sda2.in‚0 disk.sda3.in‚0.125& disk.nvme0n1.out‚53483.05517578125 disk.rbd0.out‚ 0.2978515625 disk.rbd1.in‚0 disk.rbd2.out‚457.0185546875 disk.sda.out‚ 1982.98046875 disk.sda1.in‚0 disk.rbd3.in‚0 disk.rbd1.out‚ 2.822265625 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.drbd1777.in‚0& proc.memory.availableMB‚ 987.453125 cpu‚25.720837105399877( disk.nvme0c0n1.out‚53483.05517578125 disk.dm-0.in‚ 265.6328125 disk.sda3.out‚ 1982.98046875 disk.dm-1.in‚0.125 disk.sda1.out‚0# proc.memory.rssMB‚ 2908.51171875 network.sent‚ 1.2704078e+07$ disk./.usageGB‚1057.235439300537# disk.dm-0.out‚53483.05517578125 disk.sda.in‚0.125 disk.rbd3.out‚89.89892578125$ memory_percent‚93.97306442260742 proc.cpu.threads‚17 network.recv‚ 1.90131e+06K‘ ‚(:† ÈúâÔïí°u gpu.0.gpu‚47.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚14.096755981445314* gpu.0.memoryAllocatedBytes‚ 12109021184 gpu.0.temp‚33.0 gpu.0.powerWatts‚157.385' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.477 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0X»ŸïW¡‚(:Ì ÈúâÔ†Œ©x% disk./.usageGB‚1057.2354431152344 disk.drbd1777.out‚0( disk.nvme0c0n1.out‚53483.72900390625 disk.dm-0.in‚ 265.6328125# disk.dm-0.out‚53483.72900390625 disk.dm-1.in‚0.125 disk.rbd3.in‚0 disk.dm-1.out‚ 1983.41796875 disk.rbd1.in‚0 cpu‚25.734898548577377 disk.sda2.out‚0 network.sent‚ 1.2710188e+07 network.recv‚ 1.905836e+06 disk.sda3.out‚ 1983.41796875# proc.memory.rssMB‚ 2896.39453125 disk.rbd2.out‚457.0185546875 disk.nvme0n1.in‚ 265.6328125 disk.sda2.in‚0 disk.rbd3.out‚89.89892578125 disk.rbd0.out‚ 0.2978515625$ memory_percent‚93.91124248504639 disk.drbd1777.in‚0! disk.nvme0c0n1.in‚ 265.6328125 disk.sda3.in‚0.125 disk.rbd1.out‚ 2.822265625 disk.rbd2.in‚0& disk.nvme0n1.out‚53483.72900390625 disk.sda.in‚0.125 disk.sda1.in‚0( proc.memory.availableMB‚ 997.58203125) disk./.usagePercent‚70.95195766708568 disk.sda.out‚ 1983.41796875 disk.sda1.out‚0 disk.rbd0.in‚0* proc.memory.percent‚17.678189277648926 proc.cpu.threads‚17×›”2¢- _wandbruntime‚5588  _runtime‚5588l‡ÓhBp£˜§ 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2972943186759949 9"Purity/1FRt*SRqqt0FBt/event_purity‚0.0676552951335907 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3938888609409332 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12890756130218506 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.35458236932754517 jet/accuracy_4_of_7‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.46997693181037903 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.43109607696533203 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.22384141385555267 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4315703511238098 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4038461446762085 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.40364235639572144 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 +particle/specificity‚0.9527557492256165 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.10428506880998611 particle/accuracy_2_of_5‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.2370862066745758 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3222222328186035 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09286555647850037 +particle/sensitivity‚0.3378424346446991 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.24178756773471832 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_3_of_7‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.4012279808521271 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.012763580307364464 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41439568996429443 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9906280040740967 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.4000000059604645 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_4‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.49427250027656555 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 particle/accuracy_2_of_7‚1 particle/accuracy_3_of_7‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4181841015815735 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.4388812184333801 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.044708993285894394 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994300901889801 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.34586113691329956 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.43334704637527466 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.644444465637207 +jet/accuracy_3_of_3‚0.02011602558195591 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.47805461287498474 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.18337146937847137 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.28333333134651184 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8172733187675476 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.19411255419254303 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.009370175190269947 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9865649342536926 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9932956695556641 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16074244678020477 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 trainer/global_step‚6039 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4030134081840515 9"Purity/1FRt*SRqqt*FBt/event_purity‚0.0818980261683464 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.40779221057891846 particle/accuracy_3_of_5‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.36804839968681335 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 6 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.390350878238678 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23331445455551147 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19781647622585297 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.16113987565040588 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27278777956962585 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1455891728401184 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.33060845732688904 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11921297758817673 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6333333253860474 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4470028281211853 jet/accuracy_2_of_7‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,jet/accuracy_4_of_5‚0.007895330898463726 jet/accuracy_6_of_7‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988713264465332 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004369402304291725 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.13671956956386566 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24823230504989624 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19949164986610413 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6467377543449402 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4502178728580475 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16266770660877228 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.359375 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9919141530990601 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.21604938805103302 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1811714470386505 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3516257107257843 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 jet/accuracy_5_of_7‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.496768593788147 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5176882743835449 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1979646533727646 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.41572412848472595 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 *jet/accuracy_1_of_2‚0.7196844816207886 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05476190894842148 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.021543780341744423 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.47478774189949036 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2773061692714691 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15384921431541443 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3158794045448303 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 particle/accuracy_3_of_6‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3675283193588257 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.45435166358947754 particle/accuracy_2_of_3‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5952380895614624 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3647116422653198 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.024441558867692947 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4633333384990692 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.43095237016677856 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2017434686422348 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9923194646835327 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.36375662684440613 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.45628607273101807 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 (particle/accuracy‚0.8567129969596863 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10584112256765366 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3509874939918518 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.4759085178375244 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.23982682824134827 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_4‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.14980868995189667 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.34149661660194397 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5075815320014954 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 particle/accuracy_4_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42807304859161377 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20376622676849365 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.4435451924800873 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.20707069337368011 +jet/accuracy_2_of_3‚0.23236583173274994 (particle/f_score‚0.42406150698661804 jet/accuracy_6_of_6‚0 $ _timestamp‚1.788394832787989e+09 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.46195656061172485 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_5‚1 +jet/accuracy_4_of_6‚0.07894736528396606 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.35357141494750977 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4428178668022156 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9917521476745605 6 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.150942862033844 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.23164360225200653 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6751248836517334 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 _runtime‚5588.546017598 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05287356674671173 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15213611721992493 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.003225806402042508 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.01666666753590107 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.058045659214258194 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.489857941865921 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9881480932235718 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.47926974296569824 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.8493145108222961 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18363091349601746 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_4‚1 particle/accuracy_1_of_6‚1 particle/accuracy_2_of_6‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36508646607398987 8"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.422478049993515 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.487807959318161 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_7_of_7‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13119235634803772 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09998677670955658 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6538461446762085 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 *loss/val_total_loss‚42.221174367653454 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 particle/accuracy_4_of_7‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9933123588562012 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.45435166358947754 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15213611721992493 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3349374532699585 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.397004634141922 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.48052719235420227 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 -jet/accuracy_4_of_4‚0.0040245335549116135 particle/accuracy_7_of_7‚0.5 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4651079773902893 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.471943199634552 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 _step‚489 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24611049890518188 7"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.52352374792099 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 epoch‚9 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.19491004943847656 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2184481918811798 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11594203114509583 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4023764133453369 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15305474400520325 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3062208592891693 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4093115031719208 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.44997438788414 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.011648745276033878 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.16943630576133728 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.06380462646484375 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6428571343421936 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.057984694838523865 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.14859142899513245 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22648374736309052 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9988245964050293 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4572693407535553 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9357395172119141 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4454044699668884 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19973047077655792 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4221411645412445 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1653837114572525 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.26010096073150635 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 particle/accuracy_5_of_6‚1 particle/accuracy_1_of_2‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9940556883811951 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02816438302397728 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9851189851760864 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 jet/accuracy_1_of_7‚1 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44822144508361816 7¦5O`]# Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20576037466526031 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40684691071510315 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.45368069410324097 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.01967373862862587 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.43764370679855347 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 particle/accuracy_5_of_7‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.47161465883255005 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 +jet/accuracy_2_of_4‚0.33405354619026184 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0021601382177323103 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5952380895614624 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.018534239381551743 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1445101946592331 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.22142855823040009 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4510625898838043 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.15138889849185944 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08641976118087769 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.09333334118127823 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5123456716537476 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4600694477558136 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9924979209899902 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.42066001892089844 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18714801967144012 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3771750330924988 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9891853332519531 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01295272633433342 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt0SRqqt1FBt/event_purity‚0.036296296864748 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.028609534725546837 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.46724268794059753 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4764880836009979 *jet/accuracy_1_of_4‚0.8938606977462769 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3671875 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.39035090804100037 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1608193814754486 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4599342346191406 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.22342826426029205 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4548589885234833 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4382357895374298 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.42063483595848083 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.12745098769664764 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.07014212012290955 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.15945598483085632 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2922459840774536 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.36590132117271423 +jet/accuracy_2_of_6‚0.44736841320991516 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.008025743998587132 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1651560515165329 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9905619621276855 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.00845662597566843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 jet/accuracy_5_of_6‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 particle/accuracy_2_of_2‚1 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 4 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920614361763 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3688965141773224 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.21495239436626434 -"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4375 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_1_of_7‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19385996460914612 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.15689191222190857¤¬è•\¤§ 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.19973047077655792 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4221411645412445 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1653837114572525 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.26010096073150635 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 particle/accuracy_5_of_6‚1 particle/accuracy_1_of_2‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9940556883811951 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02816438302397728 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9851189851760864 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 jet/accuracy_1_of_7‚1 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.44822144508361816 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20576037466526031 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.40684691071510315 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.45368069410324097 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.01967373862862587 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.43764370679855347 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.10000000149011612 particle/accuracy_5_of_7‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.47161465883255005 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 +jet/accuracy_2_of_4‚0.33405354619026184 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0021601382177323103 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0016977929044514894 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5952380895614624 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.018534239381551743 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1445101946592331 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.22142855823040009 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4510625898838043 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.15138889849185944 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08641976118087769 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.09333334118127823 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5123456716537476 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4600694477558136 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9924979209899902 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.42066001892089844 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18714801967144012 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3771750330924988 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9891853332519531 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01295272633433342 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt0SRqqt1FBt/event_purity‚0.036296296864748 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.028609534725546837 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.46724268794059753 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4764880836009979 *jet/accuracy_1_of_4‚0.8938606977462769 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.3671875 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.39035090804100037 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1608193814754486 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4599342346191406 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.22342826426029205 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4548589885234833 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4382357895374298 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.42063483595848083 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.12745098769664764 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.31410253047943115 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.07014212012290955 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.15945598483085632 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2922459840774536 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.36590132117271423 +jet/accuracy_2_of_6‚0.44736841320991516 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.008025743998587132 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1651560515165329 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9905619621276855 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.00845662597566843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 jet/accuracy_5_of_6‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 particle/accuracy_2_of_2‚1 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.19230769574642181 4 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920614361763 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3688965141773224 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.21495239436626434 -"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4375 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_1_of_7‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19385996460914612 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.15689191222190857 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2972943186759949 9"Purity/1FRt*SRqqt0FBt/event_purity‚0.0676552951335907 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3938888609409332 8"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.780701756477356 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12890756130218506 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.35458236932754517 jet/accuracy_4_of_7‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.375 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.46997693181037903 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.43109607696533203 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.22384141385555267 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4315703511238098 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4038461446762085 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.40364235639572144 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9335318207740784 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9983022212982178 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 +particle/specificity‚0.9527557492256165 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.10428506880998611 particle/accuracy_2_of_5‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.2370862066745758 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3222222328186035 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09286555647850037 +particle/sensitivity‚0.3378424346446991 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.24178756773471832 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_3_of_7‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.4012279808521271 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.012763580307364464 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41439568996429443 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9906280040740967 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.4000000059604645 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4583333432674408 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_4‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.49427250027656555 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2857142984867096 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 particle/accuracy_2_of_7‚1 particle/accuracy_3_of_7‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4181841015815735 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ;"Purity/1FRt2SRqqt0FBt/event_purity‚0.001612903201021254 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0416666679084301 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.4388812184333801 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.044708993285894394 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994300901889801 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.34586113691329956 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.1666666716337204 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.43334704637527466 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.644444465637207 +jet/accuracy_3_of_3‚0.02011602558195591 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.47805461287498474 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.18337146937847137 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.28333333134651184 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8172733187675476 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.19411255419254303 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.009370175190269947 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9865649342536926 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9932956695556641 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16074244678020477 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 trainer/global_step‚6039 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4030134081840515 9"Purity/1FRt*SRqqt*FBt/event_purity‚0.0818980261683464 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.40779221057891846 particle/accuracy_3_of_5‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.36804839968681335 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 6 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.390350878238678 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.23331445455551147 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9966678023338318 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19781647622585297 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.16113987565040588 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.27278777956962585 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1455891728401184 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.33060845732688904 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11921297758817673 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6333333253860474 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4470028281211853 jet/accuracy_2_of_7‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,jet/accuracy_4_of_5‚0.007895330898463726 jet/accuracy_6_of_7‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988713264465332 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.004369402304291725 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.13671956956386566 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.24823230504989624 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.19949164986610413 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6467377543449402 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4502178728580475 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4583333432674408 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16266770660877228 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.359375 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9919141530990601 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.21604938805103302 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1811714470386505 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3516257107257843 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.10000000149011612 jet/accuracy_5_of_7‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.496768593788147 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 6 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.999402642250061 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5176882743835449 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1979646533727646Æ“‰P7 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.41572412848472595 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 *jet/accuracy_1_of_2‚0.7196844816207886 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05476190894842148 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.021543780341744423 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.47478774189949036 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2773061692714691 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15384921431541443 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3158794045448303 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 particle/accuracy_3_of_6‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3675283193588257 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.45435166358947754 particle/accuracy_2_of_3‚1 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5952380895614624 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3647116422653198 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.024441558867692947 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4633333384990692 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.43095237016677856 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2017434686422348 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9923194646835327 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.36375662684440613 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.45628607273101807 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 (particle/accuracy‚0.8567129969596863 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10584112256765366 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3509874939918518 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.4759085178375244 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.23982682824134827 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_4‚1 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.14980868995189667 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.34149661660194397 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5075815320014954 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 particle/accuracy_4_of_6‚1 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.42807304859161377 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20376622676849365 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.4435451924800873 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.20707069337368011 +jet/accuracy_2_of_3‚0.23236583173274994 (particle/f_score‚0.42406150698661804 jet/accuracy_6_of_6‚0 $ _timestamp‚1.788394832787989e+09 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.46195656061172485 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_5‚1 +jet/accuracy_4_of_6‚0.07894736528396606 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.35357141494750977 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4428178668022156 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9917521476745605 6 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.150942862033844 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.23164360225200653 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6751248836517334 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 _runtime‚5588.546017598 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.780701756477356 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05287356674671173 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15213611721992493 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.003225806402042508 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.01666666753590107 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9974789619445801 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.058045659214258194 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 8"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.489857941865921 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9881480932235718 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.47926974296569824 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.8493145108222961 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18363091349601746 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_4‚1 particle/accuracy_1_of_6‚1 particle/accuracy_2_of_6‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.36508646607398987 8"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.422478049993515 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 8"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.487807959318161 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_7_of_7‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.13119235634803772 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09998677670955658 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6538461446762085 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 *loss/val_total_loss‚42.221174367653454 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_6‚1 particle/accuracy_4_of_7‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9933123588562012 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.45435166358947754 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15213611721992493 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3349374532699585 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.397004634141922 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.48052719235420227 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 -jet/accuracy_4_of_4‚0.0040245335549116135 particle/accuracy_7_of_7‚0.5 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4651079773902893 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.471943199634552 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 _step‚489 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24611049890518188 7"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.52352374792099 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 epoch‚9 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.19491004943847656 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2184481918811798 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11594203114509583 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4023764133453369 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15305474400520325 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3062208592891693 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.4093115031719208 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.44997438788414 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.011648745276033878 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.16943630576133728 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.06380462646484375 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6428571343421936 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.057984694838523865 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.14859142899513245 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.22648374736309052 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9988245964050293 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4572693407535553 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9357395172119141 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.4166666567325592 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4454044699668884é²~ŸR¥ r7y7t8hnj@ ÓúâÔàë»Æ0`Trainer.fit` stopped: `max_epochs=10` reached. 9îýÏ£¦ r7y7t8hnj ÔúâÔ€…©Ð~Epoch 9/9 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 604/604 0:08:31 • 0:00:00 1.21it/s v_num: t8hnÙ”XL%§ r7y7t8hnj ÔúâÔš¼Ð ^BÓ2¨- _wandbruntime‚5593  _runtime‚5593ŸÝ¥©‚(’Ù+