:W&Bá¾ÂƒÜM jposr5w3ª; 9 /0.29.0+7872fd9d9f86ca5c50617e19cb168122b7f959480.70.0ïø·œû‚" 5q6dpat4nu0d jposr5w3Š× jposr5w3juju11-uc-san-diegoSPAtop"š _wandb‚{}  batch_size‚512  dataset_limit‚0.2  dropout‚0.17359828058531074 epochs‚10  hidden_dim‚64 ( learning_rate‚0.00012036303295820078  num_attention_heads‚8  num_branch_encoder_layers‚3  num_embedding_layers‚10  num_encoder_layers‚2Bjjtttt_vanilla_jposr5w3b88oquqqwj)jj-spatop-sweepagent-tttt-vanilla-1-k2mn8Š ŠîâÔØø–˜š3 (H¨¸hp"3.12.10*0.29.0j linux-x86_64M 7 jposr5w3&  _wandb.runtime‚0  _runtime‚0z²ÈjÒä -Linux-6.8.0-136-generic-x86_64-with-glibc2.36CPython 3.12.10 ‰îâÔ¸ääÛ*-of*=/data/spatop/options_files/sweep_tttt_jj/config_jposr5w3.json*-l* /scratch/logs*-n*jjtttt_vanilla_jposr5w32-m spanet.trainRjacksonjulian2014@gmail.comZ /scratch/logsb)jj-spatop-sweepagent-tttt-vanilla-1-k2mn8r/opt/conda/bin/pythonº o0r21i2s77jufcvfyls8kdkf0dzlw1tf°ÏPuÉÒÀ€ˆ€’NVIDIA A100-SXM4-80GB˜¢ /€€€¥È.€ æ󢪀àãÓ½ÂS NVIDIA A100-SXM4-80GB€€€€À€6"Ampere*(GPU-78f24734-e3a1-59a7-ce1b-bb4283f77c97Ê13.2º o0r21i2s77jufcvfyls8kdkf0dzlw1tfµª½Ç2  requirements.txtÐ’Å!W jposr5w3ZF (H¨¸ (H¨¸ (hpð"3.12.10*0.29.0j linux-x86_64>/Y jposr5w3ZH (H¨¸ (H¨¸ (8hpð"3.12.10*0.29.0j linux-x86_64S–j*  jposr5w3b trainer/global_step2µ¨\  jposr5w3ZK (H¨¸ (H¨¸(8hpð"3.12.10*0.29.0j linux-x86_64:©ÜZ/  jposr5w3b*"trainer/global_step2—ä¤J  jposr5w3j9 ‹îâÔÐê«-*LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] ã|, “  jposr5w3j  ‹îâÔÀÚ—Žî â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┓ ┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃ ┡â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”┩ │ 0 │ embedding │ MultiInputVectorEmbed… │ 220 K │ train │ 0 │ │ 1 │ encoder │ JetEncoder │ 166 K │ train │ 0 │ │ 2 │ branch_decoders │ ModuleList │ 9.6 M │ train │ 0 │ │ 3 │ regression_decoder │ RegressionDecoder │ 0 │ train │ 0 │ │ 4 │ classification_decoder │ ClassificationDecoder │ 0 │ train │ 0 │ │ │ other params │ n/a │ 48 │ n/a │ n/a │ └───┴────────────────────────┴────────────────────────┴────────┴───────┴───────┘  ”‹\ jposr5w3jÊ ‹îâÔ˜š’·Trainable params: 10.0 M Non-trainable params: 86 Total params: 10.0 M Total estimated model params size (MB): 39.866 Modules in train mode: 4289 Modules in eval mode: 0 Total FLOPs: 0 عq jposr5w3j` ‹îâÔ€ª¹Pwandb: WARNING Config item 'hidden_dim' was locked by 'sweep' (ignored update). LŒ^I{ jposr5w3jj ‹îâÔð½ÂZwandb: WARNING Config item 'num_embedding_layers' was locked by 'sweep' (ignored update). €[^uy jposr5w3jh ‹îâÔÈÍÏXwandb: WARNING Config item 'num_encoder_layers' was locked by 'sweep' (ignored update). gù°*€ jposr5w3jo ‹îâÔà­×_wandb: WARNING Config item 'num_branch_encoder_layers' was locked by 'sweep' (ignored update). âz jposr5w3ji ‹îâÔˆâÝYwandb: WARNING Config item 'num_attention_heads' was locked by 'sweep' (ignored update). ”êQˆt jposr5w3jc ‹îâÔðÎæSwandb: WARNING Config item 'dataset_limit' was locked by 'sweep' (ignored update). ±A5{q jposr5w3j` ‹îâÔÇóPwandb: WARNING Config item 'batch_size' was locked by 'sweep' (ignored update). ËŽQˆt jposr5w3jc ‹îâÔèÜþSwandb: WARNING Config item 'learning_rate' was locked by 'sweep' (ignored update). €óêvn jposr5w3j] ‹îâÔü‹Mwandb: WARNING Config item 'dropout' was locked by 'sweep' (ignored update). ¹Mõm jposr5w3j\ ‹îâÔˆ½“Lwandb: WARNING Config item 'epochs' was locked by 'sweep' (ignored update). Ý¡h_ jposr5w3*Í  transformer_dim‚64  transformer_dim_scale‚2.0  initial_embedding_dim‚64  position_embedding_dim‚64 ! num_branch_embedding_layers‚3  num_jet_embedding_layers‚0  num_jet_encoder_layers‚2  num_detector_layers‚2  num_regression_layers‚3  num_classification_layers‚3  split_symmetric_attention‚1 ! transformer_activation‚"gelu"  skip_connections‚1 ( "initial_embedding_skip_connections‚1  linear_block_type‚"GRU"  transformer_type‚"Gated"  linear_activation‚"gelu"  normalization‚ "LayerNorm"  masking‚ "Filling" linear_prelu_activation‚true  use_pairwise_interactions‚0  num_pairwise_features‚4  pairwise_embedding_dim‚8  pairwise_input_source‚""  use_moe‚false  num_experts‚8  num_experts_per_tok‚2  moe_loss_scale‚0.01 F event_info_file‚2"/data/spatop/event_files/tttt/tttt_hadronic.yaml" = training_file‚+"/data/spatop/tttt_15M/h5/tttt_training.h5"  validation_file‚""  testing_file‚""  normalize_features‚1  limit_to_num_jets‚0  balance_particles‚0  balance_jets‚0 ! balance_classifications‚false  partial_events‚1  dataset_randomization‚0  train_validation_split‚0.95  limit_val_batches‚1.0  num_dataloader_workers‚0  mask_sequence_vectors‚1  combine_pair_loss‚"min"  optimizer‚"AdamW"  focal_gamma‚0.0  combinatorial_scale‚0.0 # learning_rate_warmup_epochs‚1.0  learning_rate_cycles‚1  assignment_loss_scale‚1.0  detection_loss_scale‚1.0  kl_loss_scale‚0.0  regression_loss_scale‚0.0 ! classification_loss_scale‚0.0 ! use_mass_classification‚false  mass_classes‚[]  balance_losses‚true  l2_penalty‚0.0002  gradient_clip‚10.0 num_gpu‚1  verbose_output‚false  usable_gpus‚""  trial_time‚"" $ trial_output_dir‚"./test_output"ß[×-ë jposr5w3jÙ ‹îâÔ ÷î”È/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. êt +N jposr5w3j¼ ‹îâÔø‰û”«/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. ×O+cë jposr5w3jÙ ŽîâÔ ¤à”È/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. æÝÇÒP jposr5w3j¾ ŽîâÔàµï”­/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. ÑÆ\‘‚(:‡ šîâÔåÐÝ  gpu.0.gpu‚39.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚34.0 gpu.0.powerWatts‚170.487' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0974 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0at$‚(:š šîâÔ‚âÒ disk.dm-1.out‚ 10.82421875) disk./.usagePercent‚15.61844569510838 disk.rbd1.out‚ 0.1416015625 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚0 network.sent‚33072 disk.sda.in‚0 disk.drbd2265.in‚0 disk.sda1.in‚0$ memory_percent‚80.50415515899658 cpu‚7.201162985565088 proc.cpu.threads‚17 disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.drbd2266.out‚0# proc.memory.rssMB‚ 2730.25390625% disk./.usageGB‚232.72612762451172 disk.rbd1.in‚0 disk.sda2.in‚0# disk.nvme0c0n1.out‚ 0.4658203125 disk.dm-0.out‚ 0.4658203125 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda1.out‚0 network.recv‚38240) proc.memory.percent‚16.66414737701416 disk.sda3.in‚0 disk.rbd0.out‚0! disk.nvme0n1.out‚ 0.4658203125 disk.sda.out‚ 10.82421875 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0) proc.memory.availableMB‚ 3194.19921875 disk.sda3.out‚ 10.82421875 disk.sda2.out‚0 disk.drbd1716.in‚0Ԩ¯ ‚(:† ©îâÔ¼Õô  gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚35.0 gpu.0.powerWatts‚175.085' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.017 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¯Y[¢ !‚(:– ©îâÔ’ÇŠ disk.rbd1.out‚ 0.1416015625 network.sent‚42862 network.recv‚42564% disk./.usageGB‚232.72612762451172 disk.rbd0.out‚0 disk.sda2.out‚0 disk.nvme0n1.in‚0 disk.drbd2265.in‚0 disk.sda3.in‚0$ disk.nvme0c0n1.out‚ 1.02978515625( proc.memory.percent‚16.6642427444458 disk.nvme0c0n1.in‚0) proc.memory.availableMB‚ 3193.54296875) disk./.usagePercent‚15.61844569510838 disk.dm-0.out‚ 1.02978515625 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.rbd0.in‚0" disk.nvme0n1.out‚ 1.02978515625 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.drbd2080.out‚0 disk.dm-1.out‚ 27.203125 disk.sda3.out‚ 27.203125$ memory_percent‚80.50816059112549# proc.memory.rssMB‚ 2730.26953125 cpu‚8.897643492557341 disk.dm-0.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.sda.out‚ 27.203125 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda.in‚0 disk.drbd2266.in‚0 proc.cpu.threads‚17—;Äa3"b/ lr-AdamW/pg1"trainer/global_step2:JP<Ñsˆ3#b/ lr-AdamW/pg2"trainer/global_step2:JP¯¥¥c-$) _wandbruntime‚42  _runtime‚42´ºÕ3¾%¹ & lr-AdamW/pg1‚9.764550687006354e-06 & lr-AdamW/pg2‚9.764550687006354e-06 trainer/global_step‚49 # _timestamp‚1.78839326938106e+09 _runtime‚ 42.360945449 _step‚0J&ZÀ&» _runtime‚ 42.360945449 _step‚0 & lr-AdamW/pg1‚9.764550687006354e-06 & lr-AdamW/pg2‚9.764550687006354e-06 trainer/global_step‚49 # _timestamp‚1.78839326938106e+09qtÆj'bf CDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current"trainer/global_step2:JPwµv(br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current"trainer/global_step2:JP‚íÖn)bj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed"trainer/global_step2:JPß0Ìù^*bZ 7DeviceStatsMonitor.on_train_batch_start/active.all.peak"trainer/global_step2:JP¢3…Èq+bm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current"trainer/global_step2:JP5(m3q,bm JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated"trainer/global_step2:JP«Öe‡i-be BDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed"trainer/global_step2:JP™D*ej.bf CDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated"trainer/global_step2:JP‘ãÅTq/bm JDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current"trainer/global_step2:JPö‡¿¨c0b_ ˆIt7bp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed"trainer/global_step2:JPlÛad8b` =DeviceStatsMonitor.on_train_batch_start/segment.all.allocated"trainer/global_step2:JP«GŸÎe9ba >DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed"trainer/global_step2:JPd 8k:bg DDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated"trainer/global_step2:JPÚ«åt;bp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed"trainer/global_step2:JPƒNþGkbc @DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current"trainer/global_step2:JPz¸,k?bg DDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak"trainer/global_step2:JPªop@bl IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated"trainer/global_step2:JP³@Ý­mAbi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak"trainer/global_step2:JP›´W{nBbj GDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated"trainer/global_step2:JPq7­¹aCb] :DeviceStatsMonitor.on_train_batch_start/num_oom_rejections"trainer/global_step2:JP0l³¿hDbd ADeviceStatsMonitor.on_train_batch_start/oversize_segments.current"trainer/global_step2:JP;XêgEbc @DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed"trainer/global_step2:JPp¢ÐèiFbe BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated"trainer/global_step2:JPxBÈPhGbd ADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed"trainer/global_step2:JP[ ƒqHbm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current"trainer/global_step2:JPÃÜSžrIbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated"trainer/global_step2:JPã9J»oJbk HDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current"trainer/global_step2:JP6²“lKbh EDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak"trainer/global_step2:JPœ&¾tsLbo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated"trainer/global_step2:JP“]÷†iMbe BDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current"trainer/global_step2:JPh7báeNba >DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak"trainer/global_step2:JPaùr-gObc @DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed"trainer/global_step2:JPÔF"?nPbj GDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated"trainer/global_step2:JP!2±mQbi FDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak"trainer/global_step2:JPXHÁ¦kRbg DDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current"trainer/global_step2:JP‡¥Q3jSbf CDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current"trainer/global_step2:JPHlTbh EDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed"trainer/global_step2:JP÷ Ñ_Ub[ 8DeviceStatsMonitor.on_train_batch_start/num_device_alloc"trainer/global_step2:JPXmãfVbb ?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak"trainer/global_step2:JP¿b piWbe BDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak"trainer/global_step2:JPêôÝ=nXbj GDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current"trainer/global_step2:JPÉ—_Yb[ 8DeviceStatsMonitor.on_train_batch_start/active.all.freed"trainer/global_step2:JPP¹”mZbi FDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed"trainer/global_step2:JP[¶Ís[bo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak"trainer/global_step2:JPJÇ m\bi FDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated"trainer/global_step2:JPÀvÆh]bd ADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak"trainer/global_step2:JP¹üuo^bk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed"trainer/global_step2:JPmøúp_bl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current"trainer/global_step2:JPZ¼ÕWl`bh EDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current"trainer/global_step2:JPG)SÀsabo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak"trainer/global_step2:JP‘m9WbbS 0DeviceStatsMonitor.on_train_batch_start/num_ooms"trainer/global_step2:JP;~sncbj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak"trainer/global_step2:JP¡%fidbe BDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current"trainer/global_step2:JPÖ}4febb ?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak"trainer/global_step2:JPsÅ´6lfbh EDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated"trainer/global_step2:JPóÊw4rgbn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated"trainer/global_step2:JPœMþ%ghbc @DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated"trainer/global_step2:JP²Cmiibe BDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current"trainer/global_step2:JP28à6ejba >DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak"trainer/global_step2:JP’Yƒõfkbb ?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak"trainer/global_step2:JPº æ?nlbj GDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current"trainer/global_step2:JPëRáohmbd ADeviceStatsMonitor.on_train_batch_start/active.small_pool.current"trainer/global_step2:JPÈSˆsenba >DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak"trainer/global_step2:JPuò¼âoobk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed"trainer/global_step2:JPéöH¡npbj GDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed"trainer/global_step2:JP¥0¼Xnqbj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak"trainer/global_step2:JPdμÆhrbd ADeviceStatsMonitor.on_train_batch_start/active.large_pool.current"trainer/global_step2:JPÑ_%Kosbk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed"trainer/global_step2:JP/etba >DeviceStatsMonitor.on_train_batch_start/allocation.all.current"trainer/global_step2:JPn.+Érubn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated"trainer/global_step2:JPþÍ´_vb[ 8DeviceStatsMonitor.on_train_batch_start/segment.all.peak"trainer/global_step2:JPé jwbf CDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated"trainer/global_step2:JPõû¬Ksxbo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated"trainer/global_step2:JPQ‹‘nybj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak"trainer/global_step2:JPUðúopzbl IDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current"trainer/global_step2:JPJJŒæm{bi FDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak"trainer/global_step2:JP\å~ÿk|bg DDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated"trainer/global_step2:JPC¢½Wg}bc @DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed"trainer/global_step2:JPàÁ÷\n~bj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak"trainer/global_step2:JP"1ý|jbf CDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed"trainer/global_step2:JPscßg€bb ?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak"trainer/global_step2:JPÞ÷nbi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak"trainer/global_step2:JPŸ8¼q‚bl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current"trainer/global_step2:JP—‡KŸaƒb\ 9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries"trainer/global_step2:JPWõï¦l„bg DDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated"trainer/global_step2:JPs^kÔd…b_ DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 2723984420864 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚645181 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 2719237671936 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421661184 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 35914568704 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 RGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚917366 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 2724164683776 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 2723984420864 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚407404 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 241264640 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313400320 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 35721053654 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚682182 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 trainer/global_step‚49 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 2688233075712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚1581314 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 2756541683712 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 35560987662 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 231028736 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497685 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 2649263979990 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚1599548 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚441 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22192 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 2688233075712 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚237339 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 35751345152 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚682182 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 2688250115072 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚438 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚934 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421661184 NCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚899134 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚953 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22192 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 2688233075712 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚237339 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 35751345152 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚682182 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 2688250115072 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚438 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚934 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421661184 NCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚899134 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚953 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 2723984420864 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚645181 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 2719237671936 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421661184 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 35914568704 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 RGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚917366 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 2724164683776 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 2723984420864 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚407404 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 241264640 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313400320 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 35721053654 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513ê)ˆ¹d¥b_ DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current"trainer/global_step2:JPcˆ›Lm§bh EDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current"trainer/global_step2:JP96 äi¨bd ADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current"trainer/global_step2:JPåá6yf©ba >DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated"trainer/global_step2:JP·f†bªb] :DeviceStatsMonitor.on_train_batch_end/allocation.all.freed"trainer/global_step2:JPS)Î$h«bc @DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current"trainer/global_step2:JP¿ júl¬bg DDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak"trainer/global_step2:JPôçtîV­bQ .DeviceStatsMonitor.on_train_batch_end/num_ooms"trainer/global_step2:JP®áªn®bi FDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed"trainer/global_step2:JP‰«K^¯bY 6DeviceStatsMonitor.on_train_batch_end/segment.all.peak"trainer/global_step2:JPüÄŠh°bc @DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current"trainer/global_step2:JP‚>æe±b` =DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed"trainer/global_step2:JP†j}îr²bm JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated"trainer/global_step2:JP“% ó_³bZ 7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries"trainer/global_step2:JP·»¿Wm´bh EDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current"trainer/global_step2:JPßA7 lµbg DDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated"trainer/global_step2:JPå³DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed"trainer/global_step2:JPßFÿîlËbg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak"trainer/global_step2:JPS»qÌbl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated"trainer/global_step2:JP ×<¸oÍbj GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated"trainer/global_step2:JPMˆåNoÎbj GDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated"trainer/global_step2:JP_Œ1‚pÏbk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current"trainer/global_step2:JP›E§>pÐbk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current"trainer/global_step2:JP½ìˆÇnÑbi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed"trainer/global_step2:JPxçrÒbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak"trainer/global_step2:JPb‡a^ÓbY 6DeviceStatsMonitor.on_train_batch_end/active.all.freed"trainer/global_step2:JP Š±hÔbc @DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated"trainer/global_step2:JPë»@ÔmÕbh EDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated"trainer/global_step2:JPÿlÍmÖbh EDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed"trainer/global_step2:JP˜¥j×be BDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current"trainer/global_step2:JP2 ¨jØbe BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated"trainer/global_step2:JP7y¹fÙba >DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak"trainer/global_step2:JPÌÜžfÚba >DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed"trainer/global_step2:JP‘‘&ß]ÛbX 5DeviceStatsMonitor.on_train_batch_end/num_device_free"trainer/global_step2:JPÙ°\ËrÜbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated"trainer/global_step2:JPqâƒúcÝb^ ;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak"trainer/global_step2:JPðã^çoÞbj GDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current"trainer/global_step2:JPüoX“lßbg DDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed"trainer/global_step2:JPØÍÔkwàbr ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JPØ?8dáb_ DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed"trainer/global_step2:JPxPò„qêbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated"trainer/global_step2:JPËSq‘gëbb ?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed"trainer/global_step2:JP¿¼Ñgìbb ?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current"trainer/global_step2:JPsEóP]íbX 5DeviceStatsMonitor.on_train_batch_end/active.all.peak"trainer/global_step2:JPõÀdîb_ 4¢Fmóbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak"trainer/global_step2:JP®ó¶êmôbh EDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed"trainer/global_step2:JPvYÛghõbc @DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current"trainer/global_step2:JPbu/iöbd ADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated"trainer/global_step2:JP‹W>¤k÷bf CDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed"trainer/global_step2:JP:ìu©møbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak"trainer/global_step2:JPAMF¾iùbd ADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed"trainer/global_step2:JPÔ=Ðåuúbp MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current"trainer/global_step2:JP³ÇZfûba >DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed"trainer/global_step2:JPÎ<ñðoübj GDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current"trainer/global_step2:JPÝ,6eýb` =DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak"trainer/global_step2:JPÀ<\dþb_ DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak"trainer/global_step2:JP”·ÍÈp”bk HDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current"trainer/global_step2:JPº’Í9h•bc @DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current"trainer/global_step2:JPÅ~Âi–bd ADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated"trainer/global_step2:JP\ãk—bf CDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated"trainer/global_step2:JP•ðWër˜bm JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated"trainer/global_step2:JPn­ªq™bl IDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated"trainer/global_step2:JPó\Äk^šbY 6DeviceStatsMonitor.on_train_batch_end/num_device_alloc"trainer/global_step2:JPÆùçm›bh EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed"trainer/global_step2:JPz²jUiœbd ADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current"trainer/global_step2:JP/_$Mcb^ ;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated"trainer/global_step2:JPÞ8mžbh EDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated"trainer/global_step2:JP9e³eŸb` =DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed"trainer/global_step2:JPkEƒ€. ) _wandbruntime‚43  _runtime‚433¥`mZ¡„M D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 2741640037376 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 36625328640 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 36462091776 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 231028736 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚1612802 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 2778265366016 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚695678 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚415460 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚657279 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 36428013322 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2811285326336 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 2773264404480 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚49 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚415463 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 2778085089792 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 N8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 2701888663306 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 2701711557746 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 _runtime‚ 43.055484269 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 2773239500800 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 38045825536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 2741622998016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 F:DeviceStatsMonito3ó40r.on_train_batch_end/active.all.allocated‚1631062 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚695678 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 2778265366016 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 2741640037376 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚242275 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 PEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚935384 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1631062 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1612802 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 2665443610624 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 H=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚695676 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 36625328640 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 2741622998016 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 36267947122 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313400320 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 38253784576 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚953 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚934 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 LADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚917126 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496¼ôfŽ&¢ˆM FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚49 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚415463 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 2778085089792 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 N8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 2701888663306 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 2701711557746 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 _runtime‚ 43.055484269 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 2773239500800 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 38045825536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 2741622998016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚1631062 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚695678 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 2778265366016 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 2741640037376 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚242275 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 PEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚935384 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1631062 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1612802 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 2665443610624 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 H=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚695676 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 36625328640 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 2741622998016 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 36267947122 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313400320 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 38253784576 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚953 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚934 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 LADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚917126 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 2741640037376 WGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 36625328640 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 36462091776 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 231028736 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚1612802 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 2778265366016 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚695678 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚415460 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚657279 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 36428013322 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2811285326336 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 2773264404480¤vA£b< loss/FRt1/assignment_loss"trainer/global_step2:JP¶D=WA¤b< loss/FRt2/assignment_loss"trainer/global_step2:JP‡1LvA¥b< loss/FRt4/assignment_loss"trainer/global_step2:JP ¨´@¦b; loss/FRt3/detection_loss"trainer/global_step2:JP,òX³A§b< loss/FBt3/assignment_loss"trainer/global_step2:JP>Ë A¨b< loss/FBt4/assignment_loss"trainer/global_step2:JPòøåB©b= loss/SRqqt1/detection_loss"trainer/global_step2:JPí~3Bªb= loss/SRqqt2/detection_loss"trainer/global_step2:JPB«b= loss/SRqqt3/detection_loss"trainer/global_step2:JP&†èC¬b> loss/SRqqt3/assignment_loss"trainer/global_step2:JPu,¯‰-­b( epoch"trainer/global_step2:JPf¿ÓA®b< loss/FBt2/assignment_loss"trainer/global_step2:JP _e2@¯b; loss/FRt4/detection_loss"trainer/global_step2:JP7:s@°b; loss/FBt1/detection_loss"trainer/global_step2:JP_Ä"ü@±b; loss/FBt4/detection_loss"trainer/global_step2:JPdá(ø7²b2 loss/total_loss"trainer/global_step2:JPc¯A³b< loss/FBt1/assignment_loss"trainer/global_step2:JPÕ›rÿ@´b; loss/FRt1/detection_loss"trainer/global_step2:JPÏ„VÂAµb< loss/FRt3/assignment_loss"trainer/global_step2:JPöOJ5C¶b> loss/SRqqt1/assignment_loss"trainer/global_step2:JP»Î+C·b> loss/SRqqt2/assignment_loss"trainer/global_step2:JP¥0<;B¸b= loss/SRqqt4/detection_loss"trainer/global_step2:JP”v@¹b; loss/FBt2/detection_loss"trainer/global_step2:JPãæ•ÿ@ºb; loss/FBt3/detection_loss"trainer/global_step2:JPFÌ[C»b> loss/SRqqt4/assignment_loss"trainer/global_step2:JP‚°ã]@¼b; loss/FRt2/detection_loss"trainer/global_step2:JP=4ôå.½) _wandbruntime‚43  _runtime‚43Ðç#G¾Á /loss/FRt4/assignment_loss‚7.439844802274543 /loss/FRt3/detection_loss‚2.9205657022825458 0loss/FBt3/assignment_loss‚2.7453865643703574 0loss/FBt4/assignment_loss‚2.7222015227911607 1loss/SRqqt1/detection_loss‚3.9682819961029807 1loss/SRqqt2/detection_loss‚3.5706937993617403 1loss/SRqqt3/detection_loss‚2.9817120098231134 % _timestamp‚1.7883932700770648e+09 _step‚3 1loss/SRqqt3/assignment_loss‚5.421626864818105 epoch‚0 0loss/FBt2/assignment_loss‚2.6971337593207925 -loss/FRt4/detection_loss‚5.64840020466659 /loss/FBt1/detection_loss‚7.0112475758561725 -loss/FBt4/detection_loss‚6.41252875187487 %loss/total_loss‚117.4265956600487 trainer/global_step‚49 /loss/FBt1/assignment_loss‚2.703766501866854 .loss/FRt1/detection_loss‚4.382713088506384 _runtime‚ 43.056695305 /loss/FRt3/assignment_loss‚7.218803025128548 1loss/SRqqt1/assignment_loss‚5.498452898896771 1loss/SRqqt2/assignment_loss‚5.469223366570227 0loss/SRqqt4/detection_loss‚3.114164000244013 .loss/FBt2/detection_loss‚6.232131371811285 .loss/FBt3/detection_loss‚4.994951515373859 1loss/SRqqt4/assignment_loss‚5.420588472059795 .loss/FRt2/detection_loss‚4.091304554293553 /loss/FRt1/assignment_loss‚7.392921538292607 /loss/FRt2/assignment_loss‚7.367951773461842WkŒK¿Å 0loss/SRqqt4/detection_loss‚3.114164000244013 .loss/FBt2/detection_loss‚6.232131371811285 .loss/FBt3/detection_loss‚4.994951515373859 1loss/SRqqt4/assignment_loss‚5.420588472059795 .loss/FRt2/detection_loss‚4.091304554293553 /loss/FRt1/assignment_loss‚7.392921538292607 /loss/FRt2/assignment_loss‚7.367951773461842 /loss/FRt4/assignment_loss‚7.439844802274543 /loss/FRt3/detection_loss‚2.9205657022825458 0loss/FBt3/assignment_loss‚2.7453865643703574 0loss/FBt4/assignment_loss‚2.7222015227911607 1loss/SRqqt1/detection_loss‚3.9682819961029807 1loss/SRqqt2/detection_loss‚3.5706937993617403 1loss/SRqqt3/detection_loss‚2.9817120098231134 % _timestamp‚1.7883932700770648e+09 _step‚3 1loss/SRqqt3/assignment_loss‚5.421626864818105 epoch‚0 0loss/FBt2/assignment_loss‚2.6971337593207925 -loss/FRt4/detection_loss‚5.64840020466659 /loss/FBt1/detection_loss‚7.0112475758561725 -loss/FBt4/detection_loss‚6.41252875187487 %loss/total_loss‚117.4265956600487 trainer/global_step‚49 /loss/FBt1/assignment_loss‚2.703766501866854 .loss/FRt1/detection_loss‚4.382713088506384 _runtime‚ 43.056695305 /loss/FRt3/assignment_loss‚7.218803025128548 1loss/SRqqt1/assignment_loss‚5.498452898896771 1loss/SRqqt2/assignment_loss‚5.469223366570227ÑÊßÓ‘À‚(:† ¸îâÔô€Ó  gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚179.923' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.9846 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0P®Á‚(:“ ¸îâԜ† disk.rbd1.out‚ 0.2041015625 disk.drbd2080.out‚0$ disk./.usageGB‚232.7262954711914 disk.drbd1716.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 36.2421875 disk.dm-0.in‚0! proc.memory.rssMB‚ 2730.296875 disk.sda3.in‚0 disk.nvme0c0n1.in‚0 disk.drbd2080.in‚0 disk.sda.in‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.rbd0.in‚0 proc.cpu.threads‚17 network.recv‚47839 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.sda2.out‚0 disk.rbd0.out‚0) proc.memory.availableMB‚ 3192.80859375* proc.memory.percent‚16.664409637451172* disk./.usagePercent‚15.618456959439886 disk.drbd2266.in‚0" disk.nvme0c0n1.out‚ 1.650390625 disk.dm-0.out‚ 1.650390625 disk.sda2.in‚0# memory_percent‚80.5126428604126 disk.nvme0n1.in‚0 disk.nvme0n1.out‚ 1.650390625 disk.drbd1716.out‚0 cpu‚10.304184233851812 network.sent‚49859 disk.sda.out‚ 36.2421875 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.sda3.out‚ 36.2421875¹  k’‚(:‡ ÇîâÔ  ¶  gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚162.701' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.5402 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨¥—Â(:” ÇîâÔй»  disk.dm-1.out‚46.53125 disk.sda3.out‚46.53125$ memory_percent‚80.51908016204834# proc.memory.rssMB‚ 2730.30078125 network.recv‚81545 disk.drbd2266.out‚0 disk.sda1.in‚0* proc.memory.percent‚16.664433479309082 disk.rbd1.in‚0 disk.rbd1.out‚ 0.2041015625 disk.dm-0.out‚ 2.7099609375 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 cpu‚11.495517886529306 proc.cpu.threads‚17 disk.nvme0c0n1.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚0! disk.nvme0n1.out‚ 2.7099609375 disk.sda.out‚46.53125* disk./.usagePercent‚15.618453375334408$ disk./.usageGB‚232.7262420654297 disk.drbd1716.out‚0 disk.sda3.in‚0) proc.memory.availableMB‚ 3191.75390625# disk.nvme0c0n1.out‚ 2.7099609375 disk.sda2.in‚0 disk.rbd0.out‚0 network.sent‚119154 disk.dm-0.in‚0 disk.sda2.out‚0 disk.sda.in‚0T«Sw’Ä‚(:‡ ÖîâÔÑ¯Ö  gpu.0.gpu‚50.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.828' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5656 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0éR›$Å‚(:™ ÖîâÔð«á  disk.drbd2080.in‚0 disk.sda1.in‚0 disk.dm-1.out‚ 58.86328125* proc.memory.percent‚16.664457321166992! disk.nvme0n1.out‚ 2.9482421875 disk.drbd2266.in‚0 disk.drbd2265.in‚0 cpu‚12.514436252073272 network.recv‚85591 disk.sda2.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0% disk./.usageGB‚232.72622299194336 disk.rbd1.in‚0& proc.memory.availableMB‚ 3192.71875 network.sent‚124935* disk./.usagePercent‚15.618452095296737 disk.sda3.out‚ 58.86328125 disk.nvme0n1.in‚0 disk.sda.out‚ 58.86328125 disk.sda1.out‚0 disk.dm-1.in‚0 disk.sda3.in‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda.in‚0 disk.drbd2266.out‚0 disk.rbd0.out‚0" proc.memory.rssMB‚ 2730.3046875 proc.cpu.threads‚17# disk.nvme0c0n1.out‚ 2.9482421875 disk.dm-0.in‚0 disk.dm-0.out‚ 2.9482421875 disk.drbd2265.out‚0$ memory_percent‚80.51319122314453 disk.rbd1.out‚ 0.431640625 disk.nvme0c0n1.in‚0!&Ì.Æ) _wandbruntime‚81  _runtime‚81£+µ¼ÂǼ ' lr-AdamW/pg1‚1.9728377918645493e-05 ' lr-AdamW/pg2‚1.9728377918645493e-05 trainer/global_step‚99 $ _timestamp‚1.788393308490543e+09 _runtime‚ 81.471289958 _step‚4âv´ÆÈÀ $ _timestamp‚1.788393308490543e+09 _runtime‚ 81.471289958 _step‚4 ' lr-AdamW/pg1‚1.9728377918645493e-05 ' lr-AdamW/pg2‚1.9728377918645493e-05 trainer/global_step‚99à¢þ .É) _wandbruntime‚81  _runtime‚81&~;â#ʃO GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1283196 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚472845 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚473306 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 249653248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _step‚5 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1356982 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1283660 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 5429225958400 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 5209412069376 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 71479728128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚958 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313400320 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 70936132670 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 5429045695488 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚810351 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1356980 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3177300 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 5429045695488 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚939 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 5493501073408 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 71316504576 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 5357746230272 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1802086 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 71479728128 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 5280525307398 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224749568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _runtime‚ 81.471383936 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚3159066 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 5357729190912 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚464 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 5419354018816 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 71316504576 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 5209429108736 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 71096198662 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 5357746230272 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 5429225958400 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 trainer/global_step‚99 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚1820318 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚810354 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1802086 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚461 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1283196 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚472845 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚473306 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 249653248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _step‚5 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1356982 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1283660 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 5429225958400 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 5209412069376 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 71479728128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚958 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313400320 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 70936132670 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 5429045695488 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚810351 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1356980 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3177300 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 5429045695488 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚939 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 5493501073408 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 71316504576 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 5357746230272 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1802086 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 71479728128 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 5280525307398 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224749568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _runtime‚ 81.471383936 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚3159066 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 5357729190912 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚464 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 5419354018816 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 71316504576 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 5209429108736 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 71096198662 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 5357746230272 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 5429225958400 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 trainer/global_step‚99 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚1820318 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚810354 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1802086 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚461 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 % _timestamp‚1.7883933091924965e+09 NDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3208814 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 71643092130 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 72190488064 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3190554 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 5473355847680 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1295962 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1370478 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 71803158330 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚1296435 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 231584256 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 5548483233280 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚3190554 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 74915232256 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 5411136152576 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚1370476 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1370478 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 5483146364416 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚818410 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 5483326640640 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚477552 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚959 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 212153344 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 H;‹˜&Î’M VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚959 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 212153344 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 % _timestamp‚1.7883933091924965e+09 NDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3208814 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 71643092130 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 72190488064 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3190554 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 5473355847680 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1295962 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1370478 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 71803158330 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚1296435 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 231584256 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 5548483233280 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚3190554 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 74915232256 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 5411136152576 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚1370476 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1370478 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 5483146364416 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚818410 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 5483326640640 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚477552 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264Óç.Ï) _wandbruntime‚82  _runtime‚82Õí Eп .loss/FBt1/detection_loss‚7.055770411503081 .loss/FRt1/detection_loss‚4.256027492952634 .loss/FRt4/detection_loss‚4.548013002485843 0loss/SRqqt4/detection_loss‚2.989142157467066 &loss/total_loss‚116.82612979614281 % _timestamp‚1.7883933091942673e+09 /loss/FRt4/assignment_loss‚7.397951441842157 0loss/FBt1/assignment_loss‚2.5957179163016524 0loss/SRqqt3/detection_loss‚3.026457092767462 .loss/FBt2/detection_loss‚5.044466509457146 /loss/FRt3/assignment_loss‚7.129027345350811 0loss/FBt2/assignment_loss‚2.6163879905428207 2loss/SRqqt2/assignment_loss‚5.4803893870495735 .loss/FBt3/detection_loss‚4.332016760559931 ,loss/FBt4/detection_loss‚9.3078803693926 /loss/FRt1/assignment_loss‚7.421530809747168 1loss/SRqqt1/assignment_loss‚5.500031615910905 1loss/SRqqt2/detection_loss‚3.6648015476604727 trainer/global_step‚99 _runtime‚ 82.17390067 _step‚7 /loss/FBt4/assignment_loss‚2.655672266676619 .loss/FRt2/detection_loss‚4.639620878607841 1loss/SRqqt1/detection_loss‚3.5379692639192837 epoch‚0 /loss/FRt2/assignment_loss‚7.421162805041751 1loss/SRqqt3/assignment_loss‚5.320390959398462 1loss/SRqqt4/assignment_loss‚5.418360288164257 /loss/FBt3/assignment_loss‚2.623515302187776 /loss/FRt3/detection_loss‚2.8438261811554963åýGÇÑà 2loss/SRqqt2/assignment_loss‚5.4803893870495735 .loss/FBt3/detection_loss‚4.332016760559931 ,loss/FBt4/detection_loss‚9.3078803693926 /loss/FRt1/assignment_loss‚7.421530809747168 1loss/SRqqt1/assignment_loss‚5.500031615910905 1loss/SRqqt2/detection_loss‚3.6648015476604727 trainer/global_step‚99 _runtime‚ 82.17390067 _step‚7 /loss/FBt4/assignment_loss‚2.655672266676619 .loss/FRt2/detection_loss‚4.639Jƒ/D‚620878607841 1loss/SRqqt1/detection_loss‚3.5379692639192837 epoch‚0 /loss/FRt2/assignment_loss‚7.421162805041751 1loss/SRqqt3/assignment_loss‚5.320390959398462 1loss/SRqqt4/assignment_loss‚5.418360288164257 /loss/FBt3/assignment_loss‚2.623515302187776 /loss/FRt3/detection_loss‚2.8438261811554963 .loss/FBt1/detection_loss‚7.055770411503081 .loss/FRt1/detection_loss‚4.256027492952634 .loss/FRt4/detection_loss‚4.548013002485843 0loss/SRqqt4/detection_loss‚2.989142157467066 &loss/total_loss‚116.82612979614281 % _timestamp‚1.7883933091942673e+09 /loss/FRt4/assignment_loss‚7.397951441842157 0loss/FBt1/assignment_loss‚2.5957179163016524 0loss/SRqqt3/detection_loss‚3.026457092767462 .loss/FBt2/detection_loss‚5.044466509457146 /loss/FRt3/assignment_loss‚7.129027345350811 0loss/FBt2/assignment_loss‚2.6163879905428207ô°\FÒ‚(:„ åîâÔ¹Þà  gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚165.65' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.13 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ö€²Y#Ó‚(:˜ åîâÔíéì  disk.sda3.out‚ 73.265625 disk.rbd1.out‚ 0.431640625 disk.drbd1716.in‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 network.recv‚89885) disk./.usagePercent‚15.61845670343235% disk./.usageGB‚232.72629165649414 disk.rbd1.in‚0 disk.sda2.out‚0 disk.rbd0.in‚0 disk.sda.in‚0* proc.memory.percent‚16.664481163024902 network.sent‚131596 disk.sda3.in‚0 disk.nvme0c0n1.in‚0 disk.dm-0.out‚ 3.38134765625 disk.sda2.in‚0 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.dm-1.out‚ 73.265625 disk.dm-0.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.sda.out‚ 73.265625 disk.drbd2266.out‚0 disk.sda1.out‚0) proc.memory.availableMB‚ 3193.03515625 proc.cpu.threads‚17 disk.dm-1.in‚0 disk.drbd2080.in‚0$ memory_percent‚80.51126003265381# proc.memory.rssMB‚ 2730.30859375 disk.drbd2265.in‚0$ disk.nvme0c0n1.out‚ 3.38134765625" disk.nvme0n1.out‚ 3.38134765625 cpu‚13.39807012252069´”ñÆÔ‚(:… ôîâÔº–þ  gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.62' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.924 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0óxBo)Õ‚(:ž ôîâÔÄèì $ memory_percent‚80.51371574401855 cpu‚14.17154227801113 network.sent‚181139 network.recv‚95444 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.sda1.out‚0 disk.sda3.in‚0 disk.sda2.out‚0% disk./.usageGB‚232.72631072998047 disk.drbd2266.in‚0* proc.memory.percent‚16.664481163024902* disk./.usagePercent‚15.618457983470025 disk.rbd0.out‚0 disk.sda.out‚ 87.85546875 disk.dm-1.in‚0 disk.dm-1.out‚ 87.85546875$ disk.nvme0c0n1.out‚ 3.65966796875 disk.dm-0.in‚0 disk.dm-0.out‚ 3.65966796875( proc.memory.availableMB‚ 3192.6328125 proc.cpu.threads‚17 disk.sda.in‚0 disk.drbd2265.in‚0# proc.memory.rssMB‚ 2730.30859375 disk.rbd1.in‚0 disk.drbd1716.out‚0" disk.nvme0n1.out‚ 3.65966796875 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.rbd1.out‚ 0.494140625 disk.sda2.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚0 disk.sda3.out‚ 87.85546875 disk.nvme0c0n1.in‚0$2’Ö‚(:‡ ƒïâÔÊô‰  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚170.247' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0494 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0àŸÌ(ׂ(: ƒïâÔÑ—   disk.sda3.in‚0 disk.nvme0n1.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 0.494140625 disk.dm-0.out‚ 3.89404296875 proc.cpu.threads‚17 disk.rbd0.out‚0 disk.sda.in‚0 disk.dm-1.in‚0 disk.sda3.out‚ 103.1640625 disk.dm-0.in‚0 disk.drbd2265.out‚0 disk.drbd2080.out‚0 disk.sda2.out‚0 disk.sda.out‚ 103.1640625 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.rbd1.in‚0$ disk.nvme0c0n1.out‚ 3.89404296875# proc.memory.rssMB‚ 2730.30859375 disk.sda2.in‚0$ disk./.usageGB‚232.7261962890625" disk.nvme0n1.out‚ 3.89404296875 disk.dm-1.out‚ 103.1640625 disk.drbd1716.in‚0 disk.drbd1716.out‚0& proc.memory.availableMB‚ 3192.53125* proc.memory.percent‚16.664481163024902* disk./.usagePercent‚15.618450303243996 network.recv‚100602 disk.drbd2265.in‚0 disk.nvme0c0n1.in‚0 disk.drbd2080.in‚0$ memory_percent‚80.51433563232422 cpu‚14.852808784793304 network.sent‚187834 disk.rbd0.in‚0 disk.drbd2266.out‚0¶aè0Ø+ _wandbruntime‚120  _runtime‚120M,ÐÂÙ¼ $ _timestamp‚1.788393347188701e+09 _runtime‚ 120.168457819 _step‚8 & lr-AdamW/pg1‚2.969220515028463e-05 & lr-AdamW/pg2‚2.969220515028463e-05 trainer/global_step‚149ßÆÚÀ $ _timestamp‚1.788393347188701e+09 _runtime‚ 120.168457819 _step‚8 & lr-AdamW/pg1‚2.969220515028463e-05 & lr-AdamW/pg2‚2.969220515028463e-05 trainer/global_step‚149|c§0Û+ _wandbruntime‚120  _runtime‚120˜®ç¿”'ÜŽO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 7911786634806 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 8134287233024 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2031780 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 8027242345472 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 8027225306112 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 106881664000 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2723270 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2705038 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 111281726976 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 8119420558336 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 111069560320 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4736818 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 8134287233024 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1213304 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚434 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 8230727188992 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 8134106970112 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 8134106970112 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚4755052 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1921823 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚149 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 107044887552 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1213301 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212166656 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 $ _timestamp‚1.788393347189645e+09 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 106881664000 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2031782 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 8119445462016 _step‚9 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 8027225306112 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0ÈU℘'Ý’O GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚ 120.16972976 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2705038 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 8134106970112 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 8134106970112 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚4755052 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1921823 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚149 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 107044887552 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1213301 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212166656 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 $ _timestamp‚1.788393347189645e+09 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 106881664000 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2031782 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 8119445462016 _step‚9 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 8027225306112 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 7911786634806 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 8134287233024 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2031780 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 8027242345472 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 8027225306112 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 106881664000 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2723270 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2705038 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 111281726976 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 8119420558336 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 111069560320 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4736818 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 8134287233024 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1213304 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚434 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 8230727188992 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1934482 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 112021626880 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 7857215975424 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 N6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8173447290880 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 107178303338 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8285225569280 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 7964411318122 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2741288 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8188387915264 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 8080632267776 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚713574 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243348480 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 107018237138 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2045276 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 _step‚10 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 8080615228416 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1221360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2045278 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1221363 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2723030 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2723030 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 107592410624 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚2741288 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8173422387200 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8285468917760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 8080632267776 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 107755647488 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 R@DeviceStatsMonitor.on_train_batch5?fò _end/active_bytes.all.allocated‚ 8188387915264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4768306 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4786566 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 107592410624 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 8080615228416 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 218444800 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4786566 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚149 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2045276 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1934482 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 112021626880 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 7857215975424 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 N6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8173447290880 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 107178303338 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8285225569280 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 7964411318122 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2741288 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8188387915264 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 8080632267776 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚713574 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243348480 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 107018237138 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2045276 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 _step‚10 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 8080615228416 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1221360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2045278 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1221363 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2723030 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2723030 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 107592410624 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚2741288 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8173422387200 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8285468917760 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 8080632267776 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 107755647488 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 8188387915264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4768306 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4786566 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 107592410624 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 8080615228416 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 218444800 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4786566 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚149 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2045276 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚10543047962214 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚10696738460672 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚944862 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚444 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚10401184434176 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2706580 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚10696721421312 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243361792 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚199 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚3607990 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 142446823424 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚10839168244736 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 142610046976 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚10401201473536 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3607990 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚10696721421312 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚10819512001536 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 142610046976 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 148035661824 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚10542870856862 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚10696738460672 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 % _timestamp‚1.7883933857668624e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3626222 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚ 158.750239962 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 141846488678 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚10839348507648 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚10967791025152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6314570 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚6332804 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2706580 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚10967547663360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 148254119936 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2561116 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚13 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚10839168244736 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 142610046976 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚10401201473536 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3607990 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚10696721421312 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚10819512001536 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 142610046976 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 148035661824 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 TADeviceStatsMonitor.onÛëĵl_train_batch_start/requested_bytes.all.freed‚10542870856862 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚10696738460672 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 % _timestamp‚1.7883933857668624e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3626222 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚ 158.750239962 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 141846488678 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚10839348507648 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚10967791025152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6314570 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚6332804 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2706580 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚10967547663360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 148254119936 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2561116 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚13 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 »²{Ú0î+ _wandbruntime‚159  _runtime‚159uÅhÿ°&ïªM YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚10750111343616 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚10453102157824 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1624313 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3625982 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 143320806912 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚10873513830400 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 148991023616 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚10750111343616 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚10750128382976 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚10750128382976 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245445632 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 148770481664 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2720078 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6364318 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚2573717 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233133568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10595495539970 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚10453119197184 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1624310 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚11022284312064 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6346058 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 _step‚14 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3625982 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 _runtime‚ 159.42842128 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2720076 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2573286 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚949404 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚948976 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚11022529757696 trainer/global_step‚199 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚6364318 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3644240 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚431 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1rþs€´&ð®M WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚11022284312064 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6346058 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 _step‚14 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3625982 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 _runtime‚ 159.42842128 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2720076 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2573286 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚949404 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚948976 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚11022529757696 trainer/global_step‚199 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚6364318 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3644240 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚431 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚10750111343616 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚10453102157824 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1624313 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3625982 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 143320806912 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚10873513830400 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 148991023616 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚10750111343616 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚10750128382976 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚10750128382976 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245445632 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 148770481664 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2720078 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6364318 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚2573717 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233133568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10595495539970 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚10453119197184 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1624310~zjÃ0ñ+ _wandbruntime‚159  _runtime‚159cúLòÆ /loss/FRt3/assignment_loss‚7.092055779751216 0loss/SRqqt4/assignment_loss‚5.34558504649571 &loss/total_loss‚134.99266582648326 epoch‚0 /loss/FBt4/assignment_loss‚1.764376199245453 /loss/FRt4/detection_loss‚3.8606010557607164 0loss/SRqqt1/detection_loss‚4.855570270943997 trainer/global_step‚199 .loss/FBt4/detection_loss‚34.11455979049206 $ _timestamp‚1.788393386449896e+09 /loss/FRt2/detection_loss‚3.5220575467232735 .loss/FBt2/detection_loss‚5.944466003800853 -loss/FBt3/detection_loss‚2.45112064670594 _runtime‚ 159.429508442 /loss/FRt2/assignment_loss‚7.449886583512829 2loss/SRqqt1/assignment_loss‚5.4346128933465305 0loss/SRqqt2/detection_loss‚2.113816407050437 0loss/FBt1/assignment_loss‚1.3390876278281212 .loss/FRt1/detection_loss‚5.565407387557483 1loss/SRqqt3/detection_loss‚2.2270367641961486 1loss/SRqqt4/detection_loss‚3.2603485296169916 _step‚15 /loss/FRt1/assignment_loss‚7.390965037875706 1loss/SRqqt2/assignment_loss‚5.467086706588518 2loss/SRqqt3/assignment_loss‚5.2523929819916235 /loss/FRt4/assignment_loss‚7.328510727993278 0loss/FBt2/assignment_loss‚1.4955735166194075 0loss/FBt3/assignment_loss‚1.4812852428072976 /loss/FRt3/detection_loss‚2.6861260347177685 .loss/FBt1/detection_loss‚7.5501370448619135à PóÊ /loss/FRt4/assignment_loss‚7.328510727993278 0loss/FBt2/assignment_loss‚1.4955735166194075 0loss/FBt3/assignment_loss‚1.4812852428072976 /loss/FRt3/detection_loss‚2.6861260347177685 .loss/FBt1/detection_loss‚7.550137044861913 /loss/FRt3/assignment_loss‚7.092055779751216 0loss/SRqqt4/assignment_loss‚5.34558504649571 &loss/total_loss‚134.99266582648326 epoch‚0 /loss/FBt4/assignment_loss‚1.764376199245453 /loss/FRt4/detection_loss‚3.8606010557607164 0loss/SRqqt1/detection_loss‚4.855570270943997 trainer/global_step‚199 .loss/FBt4/detection_loss‚34.11455979049206 $ _timestamp‚1.788393386449896e+09 /loss/FRt2/detection_loss‚3.5220575467232735 .loss/FBt2/detection_loss‚5.944466003800853 -loss/FBt3/detection_loss‚2.45112064670594 _runtime‚ 159.429508442 /loss/FRt2/assignment_loss‚7.449886583512829 2loss/SRqqt1/assignment_loss‚5.4346128933465305 0loss/SRqqt2/detection_loss‚2.113816407050437 0loss/FBt1/assignment_loss‚1.3390876278281212 .loss/FRt1/detection_loss‚5.565407387557483 1loss/SRqqt3/detection_loss‚2.2270367641961486 1loss/SRqqt4/detection_loss‚3.2603485296169916 _step‚15 /loss/FRt1/assignment_loss‚7.390965037875706 1loss/SRqqt2/assignment_loss‚5.467086706588518 2loss/SRqqt3/assignment_loss‚5.2523929819916235q¡¾ ô‚(:’ °ïâÔñ´£  gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚173.195' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.638999999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¼À¯9#õ‚(:˜ °ïâÔ”§ã * disk./.usagePercent‚15.618472063884406 disk.rbd1.out‚ 0.7841796875 disk.drbd2080.out‚0 disk.sda.out‚ 145.49609375 disk.sda1.in‚0 disk.nvme0c0n1.in‚0 network.sent‚249730% disk./.usageGB‚232.72652053833008 disk.dm-1.out‚ 145.49609375 disk.sda3.in‚0 disk.rbd0.out‚0 disk.rbd1.in‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0) proc.memory.availableMB‚ 3191.18359375* proc.memory.percent‚16.664505004882812 disk.drbd2265.out‚0 disk.drbd2266.out‚0 disk.sda1.out‚0 disk.dm-1.in‚0! disk.nvme0c0n1.out‚ 5.32421875 disk.drbd1716.out‚0 disk.sda.in‚0 proc.memory.rssMB‚ 2730.3125 cpu‚16.490228081163085 disk.drbd2265.in‚0 disk.sda3.out‚ 145.49609375 disk.rbd0.in‚0 disk.nvme0n1.out‚ 5.32421875 disk.dm-0.in‚0 disk.dm-0.out‚ 5.32421875 disk.sda2.out‚0$ memory_percent‚80.52256107330322 proc.cpu.threads‚17 network.recv‚115070O¼Ûèœö‚(:‘ ¿ïâÔË¹Û  gpu.0.gpu‚17.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚139.64' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.927999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ç©"÷‚(:— ¿ïâÔá£ß $ memory_percent‚80.52105903625488 disk.rbd0.in‚0 disk.sda.in‚0 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 0.7841796875 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚0 proc.memory.rssMB‚ 2730.3125* proc.memory.percent‚16.664505004882812* disk./.usagePercent‚15.618467199741257% disk./.usageGB‚232.72644805908203 disk.dm-1.in‚0 disk.dm-0.out‚ 5.55859375( proc.memory.availableMB‚ 3191.4296875 disk.dm-1.out‚ 161.19140625 disk.nvme0c0n1.in‚0! disk.nvme0c0n1.out‚ 5.55859375 disk.dm-0.in‚0 disk.drbd2266.in‚0 network.recv‚121030 disk.nvme0n1.out‚ 5.55859375 disk.sda3.out‚ 161.19140625 disk.rbd1.in‚0 disk.rbd0.out‚0 disk.sda.out‚ 161.19140625 disk.drbd2266.out‚0 proc.cpu.threads‚17 network.sent‚299953 disk.sda3.in‚0 cpu‚16.930500483089816 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.sda1.in‚0ö‘'lbø‚(:‡ ÎïâÔÍŠÜ  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782¶s º0302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.571' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.3142 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0™°ß&&ù‚(:› ÎïâÔìâ  disk.rbd0.in‚0* disk./.usagePercent‚15.618467199741257% disk./.usageGB‚232.72644805908203 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.nvme0c0n1.in‚0 cpu‚17.330446069262347 disk.sda2.in‚0 disk.nvme0n1.in‚0 disk.sda.in‚0 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.rbd0.out‚0 disk.dm-0.in‚0 disk.drbd1716.in‚0! disk.nvme0n1.out‚ 6.0517578125# disk.nvme0c0n1.out‚ 6.0517578125 disk.drbd2080.out‚0 disk.sda.out‚ 174.38671875 disk.sda3.out‚ 174.38671875# memory_percent‚80.5194616317749 network.sent‚305852 disk.dm-1.out‚ 174.38671875 proc.cpu.threads‚17 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 proc.memory.rssMB‚ 2730.3125 network.recv‚125389 disk.sda1.out‚0 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 1.01171875* proc.memory.percent‚16.664505004882812 disk.dm-0.out‚ 6.0517578125 disk.drbd2265.in‚0 disk.sda3.in‚0) proc.memory.availableMB‚ 3191.69140625 y‰0ú+ _wandbruntime‚197  _runtime‚197åUÝ Äû¾ & lr-AdamW/pg1‚4.961985961356291e-05 & lr-AdamW/pg2‚4.961985961356291e-05 trainer/global_step‚249 % _timestamp‚1.7883934244771655e+09 _runtime‚ 197.456954686 _step‚16YÜ¿éÈü & lr-AdamW/pg2‚4.961985961356291e-05 trainer/global_step‚249 % _timestamp‚1.7883934244771655e+09 _runtime‚ 197.456954686 _step‚16 & lr-AdamW/pg1‚4.961985961356291e-05wO“L0ý+ _wandbruntime‚197  _runtime‚197V‰Ø©'þ£O BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 _runtime‚ 197.461214837 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚12997070616576 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 177221633686 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚7892322 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚13366234575872 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚3381382 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 185135695872 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13519603444736 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4529174 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 177061567694 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚13366217536512 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚13704514391040 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚13544409782272 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4510942 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4529174 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚467 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224749568 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 trainer/global_step‚249 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 178011982848 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1180593 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3381380 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚13544229519360 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3199797 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1180129 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚13174309289622 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 178175206400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2019204 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 184910946304 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚13544229519360 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚7910556 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3381380 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13519628348416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0—9–­'ÿ§O MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚3381382 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 178175206400 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 249653248 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚13174132184270 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚12997087655936 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4510942 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚13366217536512 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 % _timestamp‚1.7883934244781592e+09 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2019201 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 _runtime‚ 197.461214837 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚12997070616576 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 177221633686 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚7892322 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚13366234575872 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚3381382 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 185135695872 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13519603444736 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4529174 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 177061567694 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚13366217536512 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚13704514391040 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚13544409782272 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4510942 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4529174 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚467 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224749568 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 trainer/global_step‚249 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 178011982848 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1180593 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3381380 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚13544229519360 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3199797 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1180129 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚13174309289622 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 178175206400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2019204 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 184910946304 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚13544229519360 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚7910556 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3381380 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13519628348416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚4528934 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚13048988340224 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚13598330188288 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚463 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247542784 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚13759251695104 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚13598510464512 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚18 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚13759499237888 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚7942070 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 _runtime‚ 198.153612297 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3394878 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3394876 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚13226756867378 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚7942070 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2027260 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13419624498176 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1185015 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13419624498176 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚13419607458816 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚460 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚7923810 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3212275 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 185646421504 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚7923810 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222639104 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚13049005379584 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3394878 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2027263 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4547192 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 178885966336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 177768527154 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 178722729472 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360xg‘¯9‚°M ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3212275 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 185646421504 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚7923810 [MDeviceStatsMonitor.on_tr°ÄÜ9}$ain_batch_end/inactive_split_bytes.small_pool.current‚ 222639104 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚13049005379584 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3394878 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2027263 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4547192 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 178885966336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 177768527154 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 178722729472 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚4528934 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚13048988340224 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚13598330188288 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚463 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247542784 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚13759251695104 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚13598510464512 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚18 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚13759499237888 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚7942070 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 _runtime‚ 198.153612297 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3394878 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3394876 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚13226756867378 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚7942070 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2027260 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13419624498176 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1185015 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13419624498176 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚13419607458816 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚460 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚7923810ejÏV0ƒ+ _wandbruntime‚198  _runtime‚198¯l6ØM„Ç .loss/FRt1/detection_loss‚5.039963721365169 .loss/FRt4/detection_loss‚4.393827512076027 /loss/FBt3/detection_loss‚1.2229755981253114 _step‚19 1loss/SRqqt3/assignment_loss‚4.670576994235699 1loss/SRqqt4/assignment_loss‚4.579447035519582 0loss/FBt2/assignment_loss‚0.9414709354891921 .loss/FBt1/detection_loss‚7.420409492837886 &loss/total_loss‚117.85951770787014 _runtime‚ 198.154784279 /loss/FRt1/assignment_loss‚7.458280121070751 /loss/FBt2/detection_loss‚11.215132559790757 trainer/global_step‚249 1loss/SRqqt2/assignment_loss‚4.808166412206797 /loss/FBt4/assignment_loss‚0.606614605376595 .loss/FRt3/detection_loss‚2.789578641562268 0loss/SRqqt2/detection_loss‚1.609175244626852 1loss/SRqqt3/detection_loss‚4.1866318222421866 /loss/FBt4/detection_loss‚17.831716920200147 /loss/FRt3/assignment_loss‚7.137194740528963 0loss/FBt3/assignment_loss‚0.6490191384440377 1loss/SRqqt4/detection_loss‚3.2519012378212415 /loss/FRt4/assignment_loss‚7.373802925411024 /loss/FBt1/assignment_loss‚0.621351225922505 /loss/FRt2/detection_loss‚2.7956947587430476 0loss/SRqqt1/detection_loss‚5.031970540061593 /loss/FRt2/assignment_loss‚7.405128876368205 1loss/SRqqt1/assignment_loss‚4.819486647844315 epoch‚0 % _timestamp‚1.7883934251751907e+09oŽòìQ…Ë /loss/FBt4/assignment_loss‚0.606614605376595 .loss/FRt3/detection_loss‚2.789578641562268 0loss/SRqqt2/detection_loss‚1.609175244626852 1loss/SRqqt3/detection_loss‚4.1866318222421866 /loss/FBt4/detection_loss‚17.831716920200147 /loss/FRt3/assignment_loss‚7.137194740528963 0loss/FBt3/assignment_loss‚0.6490191384440377 1loss/SRqqt4/detection_loss‚3.2519012378212415 /loss/FRt4/assignment_loss‚7.373802925411024 /loss/FBt1/assignment_loss‚0.621351225922505 /loss/FRt2/detection_loss‚2.7956947587430476 0loss/SRqqt1/detection_loss‚5.031970540061593 /loss/FRt2/assignment_loss‚7.405128876368205 1loss/SRqqt1/assignment_loss‚4.819486647844315 epoch‚0 % _timestamp‚1.7883934251751907e+09 .loss/FRt1/detection_loss‚5.039963721365169 .loss/FRt4/detection_loss‚4.393827512076027 /loss/FBt3/detection_loss‚1.2229755981253114 _step‚19 1loss/SRqqt3/assignment_loss‚4.670576994235699 1loss/SRqqt4/assignment_loss‚4.579447035519582 0loss/FBt2/assignment_loss‚0.9414709354891921 .loss/FBt1/detection_loss‚7.420409492837886 &loss/total_loss‚117.85951770787014 _runtime‚ 198.154784279 /loss/FRt1/assignment_loss‚7.458280121070751 /loss/FBt2/detection_loss‚11.215132559790757 trainer/global_step‚249 1loss/SRqqt2/assignment_loss‚4.808166412206797Þ8U¥†‚(:’ ÝïâÔÌæ…  gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚128.064' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.612799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æü#"‡‚(:— ÝïâÔð€ð  disk.dm-0.out‚ 6.5009765625 disk.drbd2080.in‚0$ memory_percent‚80.52048683166504 cpu‚17.69693299829869 disk.sda.in‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 disk.sda2.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚0! disk.nvme0n1.out‚ 6.5009765625( proc.memory.availableMB‚ 3191.5234375* proc.memory.percent‚16.664505004882812 disk.drbd2266.in‚0 disk.dm-0.in‚0 proc.memory.rssMB‚ 2730.3125 proc.cpu.threads‚17 network.recv‚129419$ disk./.usageGB‚232.7265510559082 disk.sda.out‚ 188.21484375 disk.nvme0c0n1.in‚0 disk.sda1.out‚0 disk.rbd1.in‚0# disk.nvme0c0n1.out‚ 6.5009765625 network.sent‚311622) disk./.usagePercent‚15.61847411194468 disk.drbd2265.in‚0 disk.dm-1.out‚ 188.21484375 disk.rbd1.out‚ 1.0234375 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda3.out‚ 188.21484375«!°’ˆ‚(:‡ ìïâÔ…  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚176.114' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.2228 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ad—-‰‚(:¢ ìïâÔ–Ù† disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0# proc.memory.rssMB‚ 2730.31640625% disk./.usageGB‚232.72659301757812 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2265.in‚0( proc.memory.availableMB‚ 3191.2109375 disk.sda2.in‚0 disk.sda3.in‚0 disk.nvme0c0n1.in‚0 disk.drbd1716.in‚0 disk.dm-1.in‚0 network.recv‚135794 disk.sda3.out‚ 198.09765625 disk.rbd1.in‚0 disk.sda.out‚ 198.09765625 cpu‚18.032028162340684 network.sent‚362049 disk.dm-0.out‚ 7.88330078125* proc.memory.percent‚16.664528846740723 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.nvme0n1.in‚0" disk.nvme0n1.out‚ 7.88330078125 disk.dm-1.out‚ 198.09765625* disk./.usagePercent‚15.618476928027558$ disk.nvme0c0n1.out‚ 7.88330078125 disk.rbd0.out‚0 disk.sda.in‚0 disk.sda1.out‚0 disk.drbd2266.out‚0 disk.sda1.in‚0$ memory_percent‚80.52239418029785 disk.sda2.out‚0 disk.rbd1.out‚ 1.16015625 disk.dm-0.in‚0&®¤_0Š+ _wandbruntime‚235  _runtime‚235;çk\Ä‹¾ % _timestamp‚1.7883934628870144e+09 _runtime‚ 235.866873495 _step‚20 & lr-AdamW/pg1‚5.958368684520204e-05 & lr-AdamW/pg2‚5.958368684520204e-05 trainer/global_step‚299Ï£c¯ÈŒÂ & lr-AdamW/pg1‚5.958368684520204e-05 & lr-AdamW/pg2‚5.958368684520204e-05 trainer/global_step‚299 % _timestamp‚1.7883934628870144e+09 _runtime‚ 235.866873495 _step‚20\˜ºš0+ _wandbruntime‚235  _runtime‚235#ð©'Ž£O NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16249290793984 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 213740365824 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2422154 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _step‚21 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 212596778694 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7883934628880975e+09 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚16035730691072 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚16219694887936 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16249290793984 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 213577142272 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5413894 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16441552908288 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1417127 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚16035730691072 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚16035713651712 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 221858020352 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚15805570617030 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16035713651712 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4056180 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16219719791616 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚15805393511678 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 _runtime‚ 235.871261418 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9470074 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 212436712702 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5432126 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3839718 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1417564 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15592956798976 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 trainer/global_step‚299 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16249290793984 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 213740365824 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2422154 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _step‚21 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 212596778694 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7883934628880975e+09 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚16035730691072 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚16219694887936 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16249290793984 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 213577142272 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5413894 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16441552908288 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1417127 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚16035730691072 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚16035713651712 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 221858020352 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚15805570617030 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16035713651712 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4056180 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16219719791616 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚15805393511678 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 _runtime‚ 235.871261418 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9470074 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 YHDeviceStatsMonitor.on_train_batch_start/requested_byteö×Þûm s.small_pool.freed‚ 212436712702 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5432126 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3839718 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1417564 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15592956798976 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 trainer/global_step‚299 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9519822 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5431886 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16273721620480 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16303391462912 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1421842 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚16089103574016 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1422294 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚16089120613376 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16089103574016 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 214451125760 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5450144 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3852507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚15644891561984 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚16496285100544 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚299 _step‚22 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16496536837632 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 213143672162 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚9501562 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚15858195300346 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3852052 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 % _timestamp‚1.7883934635768182e+09 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16089120613376 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 214451125760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚15858018194786 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4069676 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚455 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2430213 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 251737088 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 214287888896 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 214287888896 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0»EÃt·&’±M MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5450144 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3852507 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚15644891561984 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚16496285100544 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚299 _step‚22 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16496536837632 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 213143672162 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚9501562 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚15858195300346 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3852052 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 % _timestamp‚1.7883934635768182e+09 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16089120613376 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 214451125760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚15858018194786 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4069676 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚455 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2430213 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 251737088 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 214287888896 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 214287888896 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚9501562 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚16303571739136 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚15644874522624 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16303571739136 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4069676 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚5450144 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 222815217152 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9519822 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9519822 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5431886 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16273721620480 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16303391462912 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1421842 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚16089103574016 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1422294 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚16089120613376 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16089103574016 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 214451125760 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485ïŒÐ0“+ _wandbruntime‚236  _runtime‚236{.ØK”Å 0loss/SRqqt1/detection_loss‚5.687196116894484 epoch‚0 -loss/FRt1/assignment_loss‚7.4031401095183 .loss/FRt2/assignment_loss‚7.40752364034238 .loss/FBt3/detection_loss‚1.407341021580549 _step‚23 1loss/SRqqt3/assignment_loss‚3.261539578437805 0loss/FBt1/assignment_loss‚0.5687123022507876 0loss/FBt3/assignment_loss‚0.4142232599209935 2loss/SRqqt1/assignment_loss‚3.7768877276352475 %loss/total_loss‚135.6524272795244 /loss/FRt4/assignment_loss‚7.322364961399751 .loss/FBt2/detection_loss‚11.32821230404079 _runtime‚ 236.558131717 0loss/FBt2/assignment_loss‚0.6167732337489724 /loss/FBt4/detection_loss‚37.094046314557396 /loss/FRt3/assignment_loss‚7.138069088866071 2loss/SRqqt2/assignment_loss‚3.8832157297410825 2loss/SRqqt4/assignment_loss‚3.5494182109832764 1loss/SRqqt2/detection_loss‚2.0362410457210913 trainer/global_step‚299 .loss/FRt4/detection_loss‚4.820815692491391 0loss/SRqqt3/detection_loss‚4.098159839350142 1loss/SRqqt4/detection_loss‚2.7350706575900063 .loss/FBt1/detection_loss‚9.771831870906883 % _timestamp‚1.7883934635785046e+09 0loss/FBt4/assignment_loss‚0.5654609898726145 -loss/FRt1/detection_loss‚4.96546507615974 .loss/FRt2/detection_loss‚2.834773306095082 /loss/FRt3/detection_loss‚2.9659452014195775oÜÚ»O•É 1loss/SRqqt3/assignment_loss‚3.261539578437805 0loss/FBt1/assignment_loss‚0.5687123022507876 0loss/FBt3/assignment_loss‚0.4142232599209935 2loss/SRqqt1/assignment_loss‚3.7768877276352475 %loss/total_loss‚135.6524272795244 /loss/FRt4/assignment_loss‚7.322364961399751 .loss/FBt2/detection_loss‚11.32821230404079 _runtime‚ 236.558131717 0loss/FBt2/assignment_loss‚0.6167732337489724 /loss/FBt4/detection_loss‚37.094046314557396 /loss/FRt3/assignment_loss‚7.138069088866071 2loss/SRqqt2/assignment_loss‚3.8832157297410825 2loss/SRqqt4/assignment_loss‚3.5494182109832764 1loss/SRqqt2/detection_loss‚2.0362410457210913 trainer/global_step‚299 .loss/FRt4/detection_loss‚4.820815692491391 0loss/SRqqt3/detection_loss‚4.098159839350142 1loss/SRqqt4/detection_loss‚2.7350706575900063 .loss/FBt1/detection_loss‚9.771831870906883 % _timestamp‚1.7883934635785046e+09 0loss/FBt4/assignment_loss‚0.5654609898726145 -loss/FRt1/detection_loss‚4.96546507615974 .loss/FRt2/detection_loss‚2.834773306095082 /loss/FRt3/detection_loss‚2.9659452014195775 0loss/SRqqt1/detection_loss‚5.687196116894484 epoch‚0 -loss/FRt1/assignment_loss‚7.4031401095183 .loss/FRt2/assignment_loss‚7.40752364034238 .loss/FBt3/detection_loss‚1.407341021580549 _step‚23ºÛI$’–‚(:‡ ûïâÔ†ÿ†  gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚178.536' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.7072 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0FÑš!—‚(:– ûïâÔâ² disk.rbd0.out‚0 disk.nvme0n1.out‚ 8.287109375% disk./.usageGB‚232.72667694091797 disk.sda3.in‚0 disk.sda2.out‚0' proc.memory.availableMB‚ 3191.140625 disk.drbd2265.out‚0" proc.memory.rssMB‚ 2730.3203125" disk.nvme0c0n1.out‚ 8.287109375 disk.drbd2080.in‚0 disk.sda.in‚0 network.recv‚140672 disk.sda1.out‚0 disk.dm-1.out‚ 213.9609375 disk.sda3.out‚ 213.9609375 disk.rbd1.out‚ 1.24609375 disk.drbd1716.in‚0 disk.drbd2266.out‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 network.sent‚368647 disk.sda1.in‚0 disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.dm-0.out‚ 8.287109375 disk.drbd2265.in‚0) disk./.usagePercent‚15.61848256019331 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.sda.out‚ 213.9609375 disk.sda2.in‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0 proc.cpu.threads‚17 disk.rbd0.in‚0$ memory_percent‚80.52282333374023* proc.memory.percent‚16.664552688598633 cpu‚18.340455513876492Ã2r‘˜‚(:† ŠðâÔ¨Œ°  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚164.055' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.811 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý´j™‚(: ŠðâÔªÏÞ  network.recv‚146386 disk.sda3.in‚0 disk.rbd1.out‚ 1.41796875" proc.memory.rssMB‚ 2730.3203125# disk.nvme0c0n1.out‚ 9.1455078125 disk.rbd0.out‚0 disk.drbd2266.out‚0% disk./.usageGB‚232.72665786743164 disk.sda3.out‚234.3125 disk.sda.out‚234.3125' proc.memory.availableMB‚ 3191.078125 disk.dm-1.out‚234.3125 disk.rbd1.in‚0 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.dm-0.out‚ 9.1455078125* disk./.usagePercent‚15.618481280155638 disk.sda2.in‚0 disk.drbd2266.in‚0* proc.memory.percent‚16.664552688598633 proc.cpu.threads‚17 disk.nvme0c0n1.in‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0! disk.nvme0n1.out‚ 9.1455078125 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚0# memory_percent‚80.5232048034668 cpu‚18.624372656217094 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.dm-0.in‚0 network.sent‚418271âAeÜ’š‚(:‡ ™ðâÔõრ gpu.0.gpu‚50.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.958' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.9916 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0dÂ&¾%›‚(:š ™ðâÔ‘ãÿ  disk.drbd2265.out‚0 disk.sda1.out‚0 disk.dm-1.out‚ 245.8203125 disk.drbd2080.in‚0 proc.cpu.threads‚17 disk.sda.out‚ 245.8203125 disk.drbd2080.out‚0 network.recv‚150187 disk.sda3.out‚ 245.8203125 disk.rbd1.out‚ 1.41796875# disk.nvme0c0n1.out‚ 9.3330078125 disk.dm-0.in‚0 network.sent‚423925 disk.drbd2265.in‚0 disk.sda3.in‚0 disk.dm-0.out‚ 9.3330078125$ memory_percent‚80.52182197570801* proc.memory.percent‚16.664600372314453 disk.sda.in‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 cpu‚18.886528987809108 disk.nvme0n1.in‚0 disk.dm-1.in‚0 disk.sda2.out‚0( proc.memory.availableMB‚ 3191.3046875* disk./.usagePercent‚15.618482816200846% disk./.usageGB‚232.72668075561523 disk.rbd0.in‚0! disk.nvme0n1.out‚ 9.3330078125 disk.drbd2266.out‚0 disk.nvme0c0n1.in‚0! proc.memory.rssMB‚ 2730.328125 disk.rbd0.out‚0 disk.rbd1.in‚0¿¤‚X0œ+ _wandbruntime‚274  _runtime‚274˜Øףľ _runtime‚ 274.656249441 _step‚24 & lr-AdamW/pg1‚6.954751407684119e-05 & lr-AdamW/pg2‚6.954751407684119e-05 trainer/global_step‚349 % _timestamp‚1.7883935016764302e+09i.%Ȟ % _timestamp‚1.7883935016764302e+09 _runtime‚ 274.656249441 _step‚24 & lr-AdamW/pg1‚6.954751407684119e-05 & lr-AdamW/pg2‚6.954751407684119e-05 trainer/global_step‚349¹ ã0Ÿ+ _wandbruntime‚274  _runtime‚274Qµ¯ §O MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚18705226806272 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚18954352068608 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 249305525248 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚18188842981376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6335078 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚436 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚433 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19178789498880 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 210069504 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚18705209766912 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚18705209766912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚18705226806272 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 F@DeviceStatsMðrmþonitor.on_train_batch_start/segment.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚18919786331136 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚18188860020736 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 249142301696 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 247811857710 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 249305525248 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4478953 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 trainer/global_step‚349 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚18436831944438 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6316846 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 247971923702 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚18954352068608 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚ 274.657535449 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4730982 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1653416 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6335078 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6316846 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2825101 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11066060 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4478517 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚18919811234816 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234973184 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 258978264064 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2825104 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 258768194560 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 249142301696 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 % _timestamp‚1.7883935016774886e+09 _step‚25 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1653849 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚18954532331520 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4730980 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0p‚ÿ`±'¡«O ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚18705226806272 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚18919786331136 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚18188860020736 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 249142301696 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 247811857710 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 249305525248 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4478953 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 trainer/global_step‚349 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚18436831944438 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6316846 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 247971923702 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚18954352068608 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚ 274.657535449 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4730982 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1653416 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6335078 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6316846 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2825101 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11066060 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4478517 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚18919811234816 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234973184 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 258978264064 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2825104 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 258768194560 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 249142301696 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 % _timestamp‚1.7883935016774886e+09 _step‚25 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1653849 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 432013312 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚18954532331520 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4730980 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚18705226806272 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚18954352068608 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 249305525248 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚18188842981376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6335078 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚436 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚433 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19178789498880 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 210069504 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚18705209766912 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚18705209766912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496û…DÊ0¢+ _wandbruntime‚275  _runtime‚275<2 û¶&£°M MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4744478 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚18758599689216 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚433 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 _step‚26 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230765568 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚18240760705024 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 _runtime‚ 275.350338803 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11097574 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2833160 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19008633013760 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4491687 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2833163 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚18489456627754 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 249853048320 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚18973813063680 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 $ _timestamp‚1.788393502370163e+09 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11079314 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4744476 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚18489279522194 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 248518817170 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 250016285184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1658524 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4744476 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚18758599689216 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 trainer/global_step‚349 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 259719015424 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 259513153536 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚18240777744384 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 249853048320 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6334838 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4744478 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 205861888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 250016285184 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6353096 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚18973788160000 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚18758616728576 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4491254 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚19008452737536 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1658094 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚19008633013760 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚11079314 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6353096 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚18758616728576 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚430 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11097574 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416³AùLH¤´M ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚18758616728576 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4491254 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚19008452737536 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1658094 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚19008633013760 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚11079314 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6353096 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚18758616728576 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚430 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11097574 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4744478 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚18758599689216 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚433 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 _step‚26 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230765568 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚18240760705024 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 _runtime‚ 275.350338803 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11097574 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2833160 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19008633013760 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4491687 GADeviceSta ÝÖrtsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2833163 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚18489456627754 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 249853048320 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚18973813063680 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 $ _timestamp‚1.788393502370163e+09 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11079314 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4744476 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚18489279522194 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 248518817170 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 250016285184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1658524 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4744476 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚18758599689216 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 trainer/global_step‚349 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 259719015424 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 259513153536 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚18240777744384 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 249853048320 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6334838 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4744478 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 205861888 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 250016285184 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6353096 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚18973788160000 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471³Z0¥+ _wandbruntime‚275  _runtime‚275È—èS¦Í /loss/FBt3/detection_loss‚1.4493500016065108 /loss/FBt4/detection_loss‚17.280353941415484 /loss/FRt4/detection_loss‚3.1867750171318794 1loss/SRqqt1/detection_loss‚4.8867154332307665 .loss/FBt1/detection_loss‚7.766391286795789 epoch‚0 /loss/FRt2/assignment_loss‚7.200065928057206 2loss/SRqqt4/assignment_loss‚3.2930476002071214 1loss/FBt1/assignment_loss‚0.41621371216818015 0loss/FBt4/assignment_loss‚0.4491998220450784 /loss/SRqqt2/detection_loss‚1.65464738493679 1loss/SRqqt4/detection_loss‚2.8293250094613303 % _timestamp‚1.7883935023719568e+09 2loss/SRqqt3/assignment_loss‚3.2680268083882127 0loss/FBt3/assignment_loss‚0.4504734920081469 .loss/FRt1/detection_loss‚4.211910824348897 /loss/FRt2/detection_loss‚2.7468984136896686 1loss/SRqqt2/assignment_loss‚3.753766372348323 /loss/FRt1/assignment_loss‚7.202120251125759 1loss/SRqqt3/detection_loss‚2.3417114351167636 trainer/global_step‚349 /loss/FRt3/assignment_loss‚7.040772708494272 &loss/total_loss‚104.89613848973023 0loss/FBt2/assignment_loss‚0.4738385036587715 /loss/FRt3/detection_loss‚3.5526867634550103 .loss/FBt2/detection_loss‚8.854999613016844 _runtime‚ 275.351595986 _step‚27 /loss/FRt4/assignment_loss‚7.090672085586103 1loss/SRqqt1/assignment_loss‚3.496176081437331"=Z¹W§Ñ /loss/FRt2/assignment_loss‚7.200065928057206 2loss/SRqqt4/assignment_loss‚3.2930476002071214 1loss/FBt1/assignment_loss‚0.41621371216818015 0loss/FBt4/assignment_loss‚0.4491998220450784 /loss/SRqqt2/detection_loss‚1.65464738493679 1loss/SRqqt4/detection_loss‚2.8293250094613303 % _timestamp‚1.7883935023719568e+09 2loss/SRqqt3/assignment_loss‚3.2680268083882127 0loss/FBt3/assignment_loss‚0.4504734920081469 .loss/FRt1/detection_loss‚4.211910824348897 /loss/FRt2/detection_loss‚2.7468984136896686 1loss/SRqqt2/assignment_loss‚3.753766372348323 /loss/FRt1/assignment_loss‚7.202120251125759 1loss/SRqqt3/detection_loss‚2.3417114351167636 trainer/global_step‚349 /loss/FRt3/assignment_loss‚7.040772708494272 &loss/total_loss‚104.89613848973023 0loss/FBt2/assignment_loss‚0.4738385036587715 /loss/FRt3/detection_loss‚3.5526867634550103 .loss/FBt2/detection_loss‚8.854999613016844 _runtime‚ 275.351595986 _step‚27 /loss/FRt4/assignment_loss‚7.090672085586103 1loss/SRqqt1/assignment_loss‚3.496176081437331 /loss/FBt3/detection_loss‚1.4493500016065108 /loss/FBt4/detection_loss‚17.280353941415484 /loss/FRt4/detection_loss‚3.1867750171318794 1loss/SRqqt1/detection_loss‚4.8867154332307665 .loss/FBt1/detection_loss‚7.766391286795789 epoch‚0–ïq.¨‚(:’ ¨ðâÔ¡Š¸  gpu.0.gpu‚28.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.539' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.507799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0g7¿&©‚(:› ¨ðâÔ¶Œ´ $ disk.nvme0c0n1.out‚ 9.91943359375 disk.sda2.in‚0 disk.drbd2265.in‚0 proc.cpu.threads‚17* disk./.usagePercent‚15.618478976087832 disk.dm-0.in‚0 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2265.out‚0* proc.memory.percent‚16.664600372314453 disk.sda1.out‚0) proc.memory.availableMB‚ 3191.21484375% disk./.usageGB‚232.72662353515625 disk.sda3.in‚0 disk.nvme0c0n1.in‚0 disk.sda.out‚ 267.109375 cpu‚19.131780503017474 disk.drbd1716.in‚0 network.sent‚430876 disk.sda1.in‚0 disk.rbd1.in‚0 disk.drbd2266.in‚0 disk.nvme0n1.in‚0" disk.nvme0n1.out‚ 9.91943359375 disk.sda.in‚0! proc.memory.rssMB‚ 2730.328125 disk.rbd0.in‚0 disk.rbd0.out‚0 network.recv‚155824 disk.rbd1.out‚ 1.48046875 disk.dm-0.out‚ 9.91943359375 disk.drbd2266.out‚0$ memory_percent‚80.52237033843994 disk.drbd1716.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 267.109375 disk.sda3.out‚ 267.109375…q> ª‚(:’ ·ðâÔò‹ì  gpu.0.gpu‚45.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚35.0 gpu.0.powerWatts‚123.221' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.644199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 Ö _-«‚(:¢ ·ðâÔµË disk.dm-0.out‚10.42138671875 disk.drbd2080.out‚0# disk.nvme0n1.out‚10.42138671875 disk.drbd2265.out‚0 disk.dm-1.in‚0 proc.cpu.threads‚17 network.sent‚480993 disk.sda2.out‚0 disk.sda.out‚ 278.30859375 network.recv‚161732% disk./.usageGB‚232.72663116455078 disk.rbd1.out‚ 1.48046875 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 278.30859375$ memory_percent‚80.52399158477783) proc.memory.availableMB‚ 3190.94921875( disk./.usagePercent‚15.6184794881029% disk.nvme0c0n1.out‚10.42138671875 disk.rbd0.in‚0! proc.memory.rssMB‚ 2730.328125 disk.nvme0c0n1.in‚0 disk.drbd1716.in‚0 disk.sda.in‚0 disk.sda1.in‚0 disk.dm-1.out‚ 278.30859375 disk.drbd2266.out‚0 disk.sda1.out‚0* proc.memory.percent‚16.664600372314453 cpu‚19.358799797744165’’×£0¬+ _wandbruntime‚313  _runtime‚313Yÿ%†Ã­½ & lr-AdamW/pg1‚7.951134130848032e-05 & lr-AdamW/pg2‚7.951134130848032e-05 trainer/global_step‚399 $ _timestamp‚1.788393540066436e+09 _runtime‚ 313.046236093 _step‚28dœ*Ç®Á & lr-AdamW/pg1‚7.951134130848032e-05 & lr-AdamW/pg2‚7.951134130848032e-05 trainer/global_step‚399 $ _timestamp‚1.788393540066436e+09 _runtime‚ 313.046236093 _step‚28=\0¯+ _wandbruntime‚313  _runtime‚313Úù*z¬'°¦O M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚12643812 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚21915837867008 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 295714633728 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5405782 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5118779 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7219798 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚21374705882112 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5405780 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 284707461120 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚21659593606144 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 trainer/global_step‚399 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚21374722921472 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3228051 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 283347068710 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚455 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3228054 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚21374705882112 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 284870684672 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 284870684672 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚21619877774336 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 295935188992 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883935400674453e+09 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚21659413343232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚21659413343232 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7219798 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5118324 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚20784729163776 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚21659593606144 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚21374722921472 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5405782 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7238030 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1890273 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚21915592408064 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚21619902678016 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220555264 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416€czú°'±ªO RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚12625578 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚21659413343232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚21659413343232 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7219798 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5118324 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚20784729163776 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚21659593606144 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚21374722921472 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5405782 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7238030 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1890273 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚21915592408064 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚21619902678016 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220555264 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚12643812 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚21915837867008 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 295714633728 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5405782 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5118779 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7219798 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚21374705882112 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5405780 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 284707461120 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚21659593606144 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9351997440 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 trainer/global_step‚399 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚21374722921472 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3228051 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 283347068710 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚455 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3228054 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚21374705882112 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 284870684672 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 284870684672 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚21619877774336 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 295935188992 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7883935400674453e+09 IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 MGDeviceStatsMonitor.o»Î›Êu$n_train_batch_end/inactive_split.large_pool.current‚3 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚7237790 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚30 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5131086 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 296669507072 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7237790 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚21120717955162 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚21428095804416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚21713694288384 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚21428095804416 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7256048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚21428112843776 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚21713514012160 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚20836646887424 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚21970574013952 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚21673879603200 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚21673904506880 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚21120540849602 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚12657066 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚12675326 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 313.729347191 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚20836663926784 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3236110 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5419276 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5131537 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5131086 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 296669507072 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7237790 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚21120717955162 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚21428095804416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚21713694288384 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚21428095804416 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7256048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚21428112843776 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚21713514012160 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚20836646887424 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚21970574013952 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚21673879603200 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚21673904506880 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚21120540849602 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚12657066 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚12675326 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 313.729347191 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚20836663926784 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3236110 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5419276 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5131537 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0Hg¡0µ+ _wandbruntime‚313  _runtime‚313É‚^R¶Ì 2loss/SRqqt1/assignment_loss‚3.3820146741093815 1loss/FBt2/assignment_loss‚0.39900998770451496 0loss/FBt4/assignment_loss‚0.5093841416450838 .loss/FRt4/detection_loss‚3.427846463802068 &loss/total_loss‚109.25569670888444 epoch‚0 _runtime‚ 313.730839238 2loss/SRqqt4/assignment_loss‚3.2895597505569456 /loss/FBt3/assignment_loss‚0.331324998919359 .loss/FRt2/detection_loss‚3.959596308419503 .loss/FBt2/detection_loss‚5.320853721764353 .loss/FRt1/detection_loss‚4.856668428635933 0loss/SRqqt1/detection_loss‚4.213220941456589 /loss/FBt1/detection_loss‚12.346641109458037 /loss/FBt3/detection_loss‚1.5338918837029285 trainer/global_step‚399 /loss/FRt3/assignment_loss‚6.733150265433571 2loss/SRqqt2/assignment_loss‚3.4998312640834497 .loss/FBt4/detection_loss‚21.44601815144221 % _timestamp‚1.7883935407512352e+09 /loss/FRt2/assignment_loss‚6.708973505410803 /loss/FRt4/assignment_loss‚6.704928802407307 1loss/SRqqt3/detection_loss‚2.5492957674037173 1loss/SRqqt4/detection_loss‚3.2154766386002303 _step‚31 /loss/FRt1/assignment_loss‚6.728909929033736 1loss/SRqqt3/assignment_loss‚3.035731614760633 0loss/FBt1/assignment_loss‚0.5026314771634394 /loss/FRt3/detection_loss‚2.9908450417123857 1loss/SRqqt2/detection_loss‚1.56989184125826146"ËV·Ð /loss/FRt2/assignment_loss‚6.708973505410803 /loss/FRt4/assignment_loss‚6.704928802407307 1loss/SRqqt3/detection_loss‚2.5492957674037173 1loss/SRqqt4/detection_loss‚3.2154766386002303 _step‚31 /loss/FRt1/assignment_loss‚6.728909929033736 1loss/SRqqt3/assignment_loss‚3.035731614760633 0loss/FBt1/assignment_loss‚0.5026314771634394 /loss/FRt3/detection_loss‚2.9908450417123857 1loss/SRqqt2/detection_loss‚1.5698918412582614 2loss/SRqqt1/assignment_loss‚3.3820146741093815 1loss/FBt2/assignment_loss‚0.39900998770451496 0loss/FBt4/assignment_loss‚0.5093841416450838 .loss/FRt4/detection_loss‚3.427846463802068 &loss/total_loss‚109.25569670888444 epoch‚0 _runtime‚ 313.730839238 2loss/SRqqt4/assignment_loss‚3.2895597505569456 /loss/FBt3/assignment_loss‚0.331324998919359 .loss/FRt2/detection_loss‚3.959596308419503 .loss/FBt2/detection_loss‚5.320853721764353 .loss/FRt1/detection_loss‚4.856668428635933 0loss/SRqqt1/detection_loss‚4.213220941456589 /loss/FBt1/detection_loss‚12.346641109458037 /loss/FBt3/detection_loss‚1.5338918837029285 trainer/global_step‚399 /loss/FRt3/assignment_loss‚6.733150265433571 2loss/SRqqt2/assignment_loss‚3.4998312640834497 .loss/FBt4/detection_loss‚21.44601815144221 % _timestamp‚1.7883935407512352e+09viÌ ’¸‚(:‡ ÆðâÔÑò×  gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚172.906' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.5812 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0™$?-¹‚(:¢ ÆðâÔÿù³ disk.sda.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.rbd0.out‚0) proc.memory.availableMB‚ 3191.12890625! proc.memory.rssMB‚ 2730.328125 cpu‚19.57179815906454 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.nvme0c0n1.in‚0 disk.drbd2266.out‚0 disk.sda1.out‚0 disk.rbd1.out‚ 1.6845703125 disk.sda.out‚ 301.99609375 disk.sda3.in‚0 disk.sda3.out‚ 301.99609375 disk.dm-0.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0" disk.nvme0n1.out‚ 14.0966796875$ memory_percent‚80.52289485931396 network.recv‚166129 disk.drbd1716.out‚0% disk./.usageGB‚232.72661209106445 disk.sda1.in‚0 disk.dm-0.out‚ 14.0966796875 disk.drbd2265.in‚0 disk.dm-1.out‚ 301.99609375 disk.sda2.out‚0 disk.rbd0.in‚0 network.sent‚486773* disk./.usagePercent‚15.618478208065229 disk.dm-1.in‚0$ disk.nvme0c0n1.out‚ 14.0966796875 disk.sda2.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚0* proc.memory.percent‚16.664600372314453Ž¸«n’º‚(:‡ ÕðâÔâ–¸  gpu.0.gpu‚46.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚35.0 gpu.0.powerWatts‚131.841' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.3682 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿE¡"»‚(:— ÕðâÔ¥§ú  disk.drbd1716.out‚0 disk.drbd2080.out‚0 cpu‚19.771625112671394 proc.cpu.threads‚17 disk.sda1.out‚0 network.sent‚536214 disk.nvme0c0n1.in‚0 disk.dm-0.out‚ 14.2958984375 disk.drbd2266.out‚0 disk.sda3.out‚ 316.734375 disk.rbd1.in‚0 disk.rbd1.out‚ 1.6845703125 disk.dm-0.in‚0 disk.sda2.in‚0 disk.nvme0n1.in‚0 disk.dm-1.out‚ 316.734375* proc.memory.percent‚16.664600372314453 disk.drbd2080.in‚0 disk.dm-1.in‚0 network.recv‚171041 disk.drbd1716.in‚0 disk.drbd2265.out‚0 disk.drbd2265.in‚0 disk.sda1.in‚0* disk./.usagePercent‚15.618481536163173 disk.sda2.out‚0 disk.rbd0.in‚0 disk.sda.in‚0$ proc.memory.availableMB‚3191.125" disk.nvme0n1.out‚ 14.2958984375 disk.drbd2266.in‚0$ disk./.usageGB‚232.7266616821289$ disk.nvme0c0n1.out‚ 14.2958984375 disk.rbd0.out‚0 disk.sda3.in‚0$ memory_percent‚80.52291870117188! proc.memory.rssMB‚ 2730.328125 disk.sda.out‚ 316.734375lÑf’¼‚(:‡ äðâ滑  gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚128.353' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.6706 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ó–i1½‚(:¦ äðâÔô¹Ç disk.dm-0.out‚14.88330078125 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚0 disk.sda1.in‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 3190.13671875% disk./.usageGB‚232.72662353515625# disk.nvme0n1.out‚14.88330078125 disk.sda1.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 336.87109375 disk.rbd1.out‚ 1.7470703125% disk.nvme0c0n1.out‚14.88330078125* proc.memory.percent‚16.664600372314453 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.nvme0c0n1.in‚0 cpu‚19.957276596663387 disk.rbd1.in‚0 proc.cpu.threads‚17 disk.dm-0.in‚0 network.recv‚176563 disk.rbd0.in‚0 disk.sda3.in‚0 disk.drbd2266.in‚0 disk.sda3.out‚ 336.87109375$ memory_percent‚80.52895069122314* disk./.usagePercent‚15.618478976087832 disk.drbd1716.out‚0! proc.memory.rssMB‚ 2730.328125 network.sent‚543150 disk.sda.in‚0 disk.sda.out‚ 336.87109375 disk.drbd2266.out‚0 disk.sda2.in‚0 disk.drbd1716.in‚0Íd;à0¾+ _wandbruntime‚351  _runtime‚351xΡMÄ¿¾ trainer/global_step‚449 % _timestamp‚1.7883935784135952e+09 _runtime‚ 351.393391478 _step‚32 & lr-AdamW/pg1‚8.947516854011945e-05 & lr-AdamW/pg2‚8.947516854011945e-05ΑšÈÀ & lr-AdamW/pg1‚8.947516854011945e-05 & lr-AdamW/pg2‚8.947516854011945e-05 trainer/global_step‚449 % _timestamp‚1.7883935784135952e+09 _runtime‚ 351.393391478 _step‚32 vZ÷?0Á+ _wandbruntime‚351  _runtime‚351Ng§O IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 K?DeviceStatsMonitor.oÀŸ÷Fn_train_batch_start/active.small_pool.freed‚8122750 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚24652955631104 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6080582 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5757008 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 318722213718 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 _step‚33 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚449 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6080580 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚24364474617856 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5757482 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2126007 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 320435844096 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚23699177493902 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚24364654880768 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6080580 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚24044201997312 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚23699354599254 % _timestamp‚1.7883935784146333e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚24319994121216 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚24044201997312 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3631004 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0é਺±'ëO TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚23699177493902 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚24364654880768 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6080580 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚24044201997312 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚23699354599254 % _timestamp‚1.7883935784146333e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚24319994121216 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚24044201997312 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3631004 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 421662208 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8122750 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚24652955631104 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6080582 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5757008 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 318722213718 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18232 _step‚33 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚449 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6080580 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚24364474617856 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5757482 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2126007 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 320435844096 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416!Õ­D0Ä+ _wandbruntime‚352  _runtime‚352ßoø9·&űM UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2131127 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚24707693773824 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚24373995950080 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214250496 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 % _timestamp‚1.7883935791020231e+09 _runtime‚ 352.082136227 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6094078 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚14234818 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14253078 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6094076 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239154176 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8159000 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5770190 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 321146604032 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 333697823744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚24097608958976 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 321146604032 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3639060 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 _step‚34 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2130678 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚452 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 319269107186 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚24097591919616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6094078 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚23432550109184 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 trainer/global_step‚449 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14234818 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8140742 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5769738 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚24418755563008 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚24707454619648 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14253078 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚24418755563008 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚24097608958976 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0V‰òÿƵM D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚452 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 319269107186 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚24097591919616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6094078 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚23432550109184 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 trainer/global_step‚449 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14234818 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8140742 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5769738 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚24418755563008 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚24707454619648 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14253078 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚24418755563008 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚24097608958976 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2131127 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚24707693773824 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚24373995950080 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214250496 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 % _timestamp‚1.7883935791020231e+09 _runtime‚ 352.082136227 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6094078 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚14234818 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14253078 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6094076 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239154176 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8159000 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5770190 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 321146604032 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 333697823744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚24097608958976 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 321146604032 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3639060 KDDeviceStatsMonitor.on_train_batÜj®¼ch_end/inactive_split.large_pool.peak‚28 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 _step‚34 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2130678 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622"z°Ï©0Ç+ _wandbruntime‚352  _runtime‚352 ‚úÿPÈÊ 2loss/SRqqt4/assignment_loss‚3.2443959102630617 .loss/FRt2/detection_loss‚3.370773010493554 /loss/FRt3/detection_loss‚2.9637026339769363 epoch‚0 % _timestamp‚1.7883935791037495e+09 /loss/FRt2/assignment_loss‚6.737936604883253 1loss/SRqqt3/assignment_loss‚3.067014927505165 /loss/FBt2/assignment_loss‚0.357177530034581 /loss/FBt4/assignment_loss‚0.464194652888303 .loss/FBt1/detection_loss‚9.014514913430085 .loss/FBt3/detection_loss‚1.803043161113688 %loss/total_loss‚90.60886024212274 _step‚35 1loss/FBt1/assignment_loss‚0.49734715537822527 0loss/FBt3/assignment_loss‚0.5079194139951552 /loss/FRt4/detection_loss‚2.9189278894217217 .loss/SRqqt1/detection_loss‚3.7550447147067 _runtime‚ 352.083449296 /loss/FRt4/assignment_loss‚6.682211403534791 2loss/SRqqt1/assignment_loss‚3.1120103248735753 1loss/SRqqt2/detection_loss‚1.6079697845091614 1loss/SRqqt3/detection_loss‚2.9540295573852715 /loss/FRt3/assignment_loss‚6.677408143093712 2loss/SRqqt2/assignment_loss‚3.4635432763975493 /loss/FBt2/detection_loss‚3.2432161167574427 0loss/FRt1/assignment_loss‚6.6864318032013745 1loss/SRqqt4/detection_loss‚2.5041402758657934 trainer/global_step‚449 .loss/FRt1/detection_loss‚4.417602810028352 /loss/FBt4/detection_loss‚10.558304228385289·"‘iTÉÎ /loss/FRt2/assignment_loss‚6.737936604883253 1loss/SRqqt3/assignment_loss‚3.067014927505165 /loss/FBt2/assignment_loss‚0.357177530034581 /loss/FBt4/assignment_loss‚0.464194652888303 .loss/FBt1/detection_loss‚9.014514913430085 .loss/FBt3/detection_loss‚1.803043161113688 %loss/total_loss‚90.60886024212274 _step‚35 1loss/FBt1/assignment_loss‚0.49734715537822527 0loss/FBt3/assignment_loss‚0.5079194139951552 /loss/FRt4/detection_loss‚2.9189278894217217 .loss/SRqqt1/detection_loss‚3.7550447147067 _runtime‚ 352.083449296 /loss/FRt4/assignment_loss‚6.682211403534791 2loss/SRqqt1/assignment_loss‚3.1120103248735753 1loss/SRqqt2/detection_loss‚1.6079697845091614 1loss/SRqqt3/detection_loss‚2.9540295573852715 /loss/FRt3/assignment_loss‚6.677408143093712 2loss/SRqqt2/assignment_loss‚3.4635432763975493 /loss/FBt2/detection_loss‚3.2432161167574427 0loss/FRt1/assignment_loss‚6.6864318032013745 1loss/SRqqt4/detection_loss‚2.5041402758657934 trainer/global_step‚449 .loss/FRt1/detection_loss‚4.417602810028352 /loss/FBt4/detection_loss‚10.558304228385289 2loss/SRqqt4/assignment_loss‚3.2443959102630617 .loss/FRt2/detection_loss‚3.370773010493554 /loss/FRt3/detection_loss‚2.9637026339769363 epoch‚0 % _timestamp‚1.7883935791037495e+09#V àè‘Ê‚(:† óðâÔ›ôö  gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚37.0 gpu.0.powerWatts‚175.325' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.065 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚04E\+Ë‚(:  óðâÔ‚ýÔ  disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1716.out‚0# proc.memory.rssMB‚ 2730.33203125 disk.sda2.out‚0* disk./.usagePercent‚15.618496640607693 disk.drbd2080.in‚0 disk.sda.in‚0' proc.memory.availableMB‚ 3191.015625 network.sent‚549730 disk.sda2.in‚0 disk.drbd2080.out‚0# disk.nvme0n1.out‚15.12939453125 disk.drbd2265.out‚0* proc.memory.percent‚16.664624214172363 network.recv‚181462 disk.sda3.in‚0 disk.dm-0.out‚15.12939453125 disk.drbd1716.in‚0% disk./.usageGB‚232.72688674926758 disk.sda3.out‚ 347.671875 disk.rbd1.in‚0 cpu‚20.132281416320794 proc.cpu.threads‚17 disk.rbd1.out‚ 1.7470703125% disk.nvme0c0n1.out‚15.12939453125 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.rbd0.out‚0 disk.sda.out‚ 347.671875 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 347.671875$ memory_percent‚80.52358627319336@ó’Ì‚(:‡ ‚ñâÔ¯ôê  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚124.046' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.8092 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0çõ0Í‚(:¥ ‚ñâÔõÀ„  network.sent‚599240 disk.sda.in‚0 disk.sda.out‚ 358.94921875 disk.sda3.out‚ 358.94921875 disk.drbd1716.out‚0 cpu‚20.29694886049723 network.recv‚186867% disk./.usageGB‚232.72686767578125 disk.drbd2266.in‚0 disk.nvme0c0n1.in‚0 disk.dm-0.out‚15.90771484375$ memory_percent‚80.52229881286621# proc.memory.rssMB‚ 2730.33203125 proc.cpu.threads‚17 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 358.94921875 disk.sda3.in‚0* proc.memory.percent‚16.664624214172363* disk./.usagePercent‚15.618495360570023 disk.drbd2080.out‚0 disk.rbd1.out‚ 1.888671875 disk.drbd2080.in‚0 disk.nvme0n1.in‚0# disk.nvme0n1.out‚15.90771484375 disk.rbd1.in‚0 disk.dm-0.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0% disk.nvme0c0n1.out‚15.90771484375 disk.sda2.in‚0( proc.memory.availableMB‚ 3191.2265625 disk.sda1.in‚0 disk.dm-1.in‚0ñÖF0Î+ _wandbruntime‚389  _runtime‚389.š1ŒÂϼ _step‚36 % lr-AdamW/pg1‚9.94389957717586e-05 % lr-AdamW/pg2‚9.94389957717586e-05 trainer/global_step‚499 % _timestamp‚1.7883936166889882e+09 _runtime‚ 389.668786403[¼6/ÆÐÀ _step‚36 % lr-AdamW/pg1‚9.94389957717586e-05 % lr-AdamW/pg2‚9.94389957717586e-05 trainer/global_step‚499 % _timestamp‚1.7883936166889882e+09 _runtime‚ 389.668786403$4W0Ñ+ _wandbruntime‚389  _runtime‚389 ÉäÚ­'Ò§O NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 _step‚37 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚26713698112512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9043934 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243361792 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15799316 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4033954 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 369917456896 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6755382 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚25976501528576 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9025702 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚15781082 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6755380 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 356001003520 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚26330438821310 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚25976518567936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚26713715151872 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 355837779968 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚27069535892480 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚27069716155392 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2363883 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚27020060660736 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚27069535892480 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9043934 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚26330615926662 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9025702ÅÅO±'Ó«O IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚26713698112512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9043934 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243361792 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15799316 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4033954 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 369917456896 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6755382 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚25976501528576 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9025702 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚15781082 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6755380 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 356001003520 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22471 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163223552 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚26330438821310 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚25976518567936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚26713715151872 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 355837779968 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚27069535892480 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚27069716155392 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2363883 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚27020060660736 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚27069535892480 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9043934 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚26330615926662 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9025702 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 _step‚37 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194%¢5mœ’Ô‚(:‡ ‘ñâԱɣ  gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚125.106' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.0212 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ýj”.Õ‚(:£ ‘ñâÔÿ€Í network.recv‚190616 disk.rbd1.out‚ 1.888671875 disk.sda.out‚ 381.27734375 disk.drbd2266.out‚0 disk.sda3.out‚ 381.27734375# memory_percent‚80.5208683013916* proc.memory.percent‚16.664648056030273 network.sent‚604905* disk./.usagePercent‚15.618498176652897% disk.nvme0c0n1.out‚16.21240234375 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.sda.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0% disk./.usageGB‚232.72690963745117 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚0" proc.memory.rssMB‚ 2730.3359375 proc.cpu.threads‚17 disk.dm-0.in‚0 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 cpu‚20.45200909049228 disk.nvme0c0n1.in‚0 disk.rbd0.out‚0# disk.nvme0n1.out‚16.21240234375 disk.dm-1.out‚ 381.27734375 disk.drbd2080.out‚0 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.dm-0.out‚16.21240234375 disk.drbd2080.in‚0( proc.memory.availableMB‚ 3191.4609375 disk.rbd1.in‚00ª0Ö+ _wandbruntime‚390  _runtime‚390Uú‚> ×±M ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9043694 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15830830 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6768878 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9061952 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4042013 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚26383240609978 trainer/global_step‚499 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 _step‚38 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚27444494723072 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 370432233472 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9043694 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚15812570 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 354804318394 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 TCDeviceStatsMonitor.on_train_batch_end/active_byÆ;z ytes.small_pool.freed‚ 356548526592 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6768876 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15830830 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚26028419252224 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6410136 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚27074062489600 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6410605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0xJo»&صM WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚15812570 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 354804318394 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 356548526592 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6768876 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15830830 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚26028419252224 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6410136 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚27074062489600 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6410605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9043694 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15830830 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6768878 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9061952 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4042013 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚26383240609978 trainer/global_step‚499 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 _step‚38 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚27444494723072 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 370432233472 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9043694 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568&àfž<0Ù+ _wandbruntime‚390  _runtime‚390»¾+QÚË 1loss/FBt2/assignment_loss‚0.30819969845861356 0loss/FBt4/assignment_loss‚0.5725730121623547 1loss/SRqqt1/detection_loss‚3.4971083765954116 .loss/FRt3/assignment_loss‚6.65785884275669 1loss/SRqqt2/assignment_loss‚3.435093410553471 1loss/SRqqt4/assignment_loss‚3.311138474664023 .loss/FBt3/detection_loss‚5.023027717275369 _runtime‚ 390.361727109 _step‚39 0loss/FRt4/assignment_loss‚6.6412588534968915 0loss/SRqqt3/assignment_loss‚3.09724942105157 1loss/SRqqt2/detection_loss‚3.3219672086627132 trainer/global_step‚499 /loss/FRt2/assignment_loss‚6.676891098217088 /loss/FRt3/detection_loss‚2.8747093539412427 1loss/SRqqt3/detection_loss‚1.5859840754020427 % _timestamp‚1.7883936173821442e+09 0loss/FBt1/assignment_loss‚0.5133120818592777 1loss/FBt3/assignment_loss‚0.48443345303336777 .loss/FRt2/detection_loss‚3.298548935931556 .loss/FBt1/detection_loss‚9.826977783983404 /loss/FBt4/detection_loss‚14.721988046453113 2loss/SRqqt1/assignment_loss‚3.0264428760228532 /loss/FRt4/detection_loss‚3.0976834136365663 /loss/FBt2/detection_loss‚2.0272356005182037 %loss/total_loss‚97.92599122403932 /loss/FRt1/assignment_loss‚6.726564884185791 .loss/FRt1/detection_loss‚4.749038055539131 0loss/SRqqt4/detection_loss‚2.450706549638579 epoch‚0b ßéUÛÏ %loss/total_loss‚97.92599122403932 /loss/FRt1/assignment_loss‚6.726564884185791 .loss/FRt1/detection_loss‚4.749038055539131 0loss/SRqqt4/detection_loss‚2.450706549638579 epoch‚0 1loss/FBt2/assignment_loss‚0.30819969845861356 0loss/FBt4/assignment_loss‚0.5725730121623547 1loss/SRqqt1/detection_loss‚3.4971083765954116 .loss/FRt3/assignment_loss‚6.65785884275669 1loss/SRqqt2/assignment_loss‚3.435093410553471 1loss/SRqqt4/assignment_loss‚3.311138474664023 .loss/FBt3/detection_loss‚5.023027717275369 _runtime‚ 390.361727109 _step‚39 0loss/FRt4/assignment_loss‚6.6412588534968915 0loss/SRqqt3/assignment_loss‚3.09724942105157 1loss/SRqqt2/detection_loss‚3.3219672086627132 trainer/global_step‚499 /loss/FRt2/assignment_loss‚6.676891098217088 /loss/FRt3/detection_loss‚2.8747093539412427 1loss/SRqqt3/detection_loss‚1.5859840754020427 % _timestamp‚1.7883936173821442e+09 0loss/FBt1/assignment_loss‚0.5133120818592777 1loss/FBt3/assignment_loss‚0.48443345303336777 .loss/FRt2/detection_loss‚3.298548935931556 .loss/FBt1/detection_loss‚9.826977783983404 /loss/FBt4/detection_loss‚14.721988046453113 2loss/SRqqt1/assignment_loss‚3.0264428760228532 /loss/FRt4/detection_loss‚3.0976834136365663 /loss/FBt2/detection_loss‚2.0272356005182037'­K¢Ü‚(:…  ñâÔÝÒÏ  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.09' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.818 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ˆZÝ.Ý‚(:£  ñâÔ ¢ë  disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 396.35546875 disk.drbd2080.out‚0" proc.memory.rssMB‚ 2730.3359375 network.recv‚196909% disk./.usageGB‚232.72689056396484 disk.nvme0c0n1.in‚0 disk.drbd2080.in‚0$ memory_percent‚80.52432537078857 disk.drbd2266.out‚0 disk.dm-1.in‚0 disk.dm-0.out‚ 16.7783203125 disk.sda2.in‚0* disk./.usagePercent‚15.618496896615227 disk.rbd1.in‚0 disk.sda2.out‚0 disk.drbd1716.in‚0* proc.memory.percent‚16.664648056030273 cpu‚20.598154889435612 disk.rbd0.out‚0 disk.nvme0n1.in‚0" disk.nvme0n1.out‚ 16.7783203125 disk.dm-1.out‚ 396.35546875 disk.drbd1716.out‚0 proc.cpu.threads‚17 disk.sda1.in‚0 disk.dm-0.in‚0) proc.memory.availableMB‚ 3190.89453125 network.sent‚644029 disk.sda.in‚0 disk.sda.out‚ 396.35546875 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 1.951171875$ disk.nvme0c0n1.out‚ 16.7783203125Æ ¼Þ‚(:… ¯ñâÔ·ÍŒ  gpu.0.gpu‚25.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚174.26' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.852 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0M½¼ß‚(:“ ¯ñâÔÖ¬ü disk.sda3.out‚412.9375 disk.rbd1.out‚ 1.951171875 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.sda2.in‚0$ memory_percent‚80.52139282226562 proc.cpu.threads‚17 disk.sda.in‚0 disk.dm-1.out‚412.9375 disk.sda2.out‚0* disk./.usagePercent‚15.618495104562488 disk.rbd0.in‚0 disk.rbd0.out‚0" disk.nvme0n1.out‚ 16.9423828125 disk.sda3.in‚0* proc.memory.percent‚16.664671897888184 network.sent‚683541% disk./.usageGB‚232.72686386108398 disk.dm-0.out‚ 16.9423828125 disk.drbd1716.in‚0 cpu‚20.737593911405135 disk.drbd2266.out‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.rbd1.in‚0$ proc.memory.availableMB‚3191.375$ disk.nvme0c0n1.out‚ 16.9423828125 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda.out‚412.9375 disk.drbd2266.in‚0 disk.drbd1716.out‚0# proc.memory.rssMB‚ 2730.33984375 network.recv‚201934 disk.nvme0n1.in‚0 disk.sda1.in‚0Ɇ÷q0à+ _wandbruntime‚428  _runtime‚428-0ÊÆáÀ % _timestamp‚1.7883936551898766e+09 _runtime‚ 428.169659877 _step‚40 ' lr-AdamW/pg1‚0.00010940282300339774 ' lr-AdamW/pg2‚0.00010940282300339774 trainer/global_step‚549`øZÊâÄ ' lr-AdamW/pg1‚0.00010940282300339774 ' lr-AdamW/pg2‚0.00010940282300339774 trainer/global_step‚549 % _timestamp‚1.7883936551898766e+09 _runtime‚ 428.169659877 _step‚40(f›|Ê0ã+ _wandbruntime‚428  _runtime‚428ˆTSì8%ä§O LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚28572387710976 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚463 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚28961877254070 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 389312437742 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9946886 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚29383211267072 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220555264 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚28572404750336 trainer/global_step‚549 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7430180 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7430182 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245458944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 391566162944 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚29774777430016 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 391566162944 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7430180 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 % _timestamp‚1.7883936551908736e+09 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚29383194227712 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚30126778733056 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 389472503734 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚28961700148718 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7038809 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17377068 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2601905 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚41 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 391402939392 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7038346 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚460 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9928654 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚29383194227712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚29774597167104 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29720177007616 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚29383211267072 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9928654 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9946886 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4436901 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 421662208 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚28572387710976 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚463 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚28961877254070 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 389312437742 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9946886 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚29383211267072 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220555264 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚28572404750336 trainer/global_step‚549 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7430180 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7430182 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245458944 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 391566162944 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚29774777430016 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 391566162944 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177105352 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7430180 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 432013312 % _timestamp‚1.7883936551908736e+09 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚29383194227712 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚30126778733056 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 389472503734 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9573498880 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚28961700148718 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7038809 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17377068 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2601905 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚41 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22471 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163223552 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 391402939392 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7038346 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚460 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18232 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9928654 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 233681408 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚29383194227712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚622 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚622 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚29774597167104 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29720177007616 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056814485 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚29383211267072 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9928654)Ö»Ý0æ+ _wandbruntime‚428  _runtime‚428û¼P{·&ç±M @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17390322 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9946646 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚29014501937386 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚447 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2606107 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 390019397202 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚29828697836032 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7051067 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚29774178836480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚28624305434624 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 392113686016 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚9964904 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9964904 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚30181519037440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚29774153932800 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚29436584150016 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚28624322473984 % _timestamp‚1.7883936558691833e+09 _step‚42 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 390179463402 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7443678 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7443676 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9946646 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚29436584150016 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4444960 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 trainer/global_step‚549 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 392113686016 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2606554 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17408582 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17408582 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚29014324831826 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 _runtime‚ 428.849767382 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 392276922880 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 392276922880 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7443676 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚29436601189376 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7443678 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 407365104640 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7051517 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚29828878112256 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 407587743744 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247542784 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222639104 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚29828878112256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚450 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4444963 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚30181766580224 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 û¸†»&èµM E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 392276922880 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7443676 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚29436601189376 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7443678 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 407365104640 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7051517 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚29828878112256 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 407587743744 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247542784 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222639104 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚29828878112256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚450 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4444963 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚30181766580224 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418497693 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17390322 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9946646 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180276224 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚29014501937386 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚447 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2606107 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 390019397202 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚29828697836032 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 421662208 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7051067 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚29774178836480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚28624305434624 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 392113686016 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚9964904 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9964904 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚30181519037440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚29774153932800 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚29436584150016 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚28624322473984 % _timestamp‚1.7883936558691833e+09 _step‚42 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 390179463402 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚206 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7443678 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7443676 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9946646 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚29436584150016 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4444960 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 trainer/global_step‚549 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 392113686016 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2606554 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17408582 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17408582 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚29014324831826 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 _runtime‚ 428.849767382 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 392276922880 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 ODeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚32420243547136 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚599 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10831606 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚32479658441728 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10849838 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7679818 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚32052690342912 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚450 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214263808 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚10849838 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚32420268450816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 424687582750 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180262912 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32479658441728 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8104982 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8104980 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7679365 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 426968098816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32052690342912 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32479658441728 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8104982 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8104980 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180262912 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7679365 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 426968098816 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22194 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9351997440 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32052690342912 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚622 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚622 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚32420243547136 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160065992 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9573498880 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 432013312 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚599 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10831606 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚32479658441728 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10849838 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7679818 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚206 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚32052690342912 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22194 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚450 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214263808 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚206 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18234 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚206 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚10849838 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418497693 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18234 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚32420268450816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 424687582750 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568-tÝYT0ø+ _wandbruntime‚467  _runtime‚467G4÷'ùµM NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10867856 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚32106097304576 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10867856 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚32533939386880 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10849598 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 444545452032 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 427842082304 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7692089 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚32106097304576 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 427678845440 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚32106080265216 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 444329104384 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚32106080265216 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚8118476 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8118478 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚32474270279680 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8118478 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31645763264794 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚31220191617024 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚32918815731712 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4847913 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚31220208656384 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 G:DeviceStatsMonitor.on_train_batch_}÷§” end/allocation.all.freed‚18968074 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 _step‚46 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚32533759110656 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚18986334 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7692541 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚449 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 425394542210 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚18986334 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚32533939386880 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2844179 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241251328 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚32474245376000 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚18968074 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 427678845440 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2844628 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216347648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 427842082304 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0*躮¿&ú¹M REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚8118476 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8118478 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18258 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚32474270279680 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 432013312 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚206 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8118478 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31645763264794 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚31220191617024 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160066200 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚622 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056814485 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚206 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚32918815731712 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180276224 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22471 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4847913 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚31220208656384 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚622 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚18968074 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 _step‚46 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚32533759110656 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163236864 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚18986334 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7692541 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚449 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 425394542210 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 432013312 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22471 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚18986334 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9573498880 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚32533939386880 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177105560 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18258 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2844179 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241251328 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚32474245376000 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9573498880 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18260 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚18968074 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 427678845440 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2844628 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216347648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 427842082304 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10867856 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9351997440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚32106097304576 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10867856 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22194 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚622 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163236864 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚32533939386880 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10849598 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 444545452032 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 233681408 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9573498880 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 427842082304 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚622 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7692089 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 432013312 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚32106097304576 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9351997440 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 421662208 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 427678845440 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚32106080265216 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 444329104384 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚32106080265216.ÛÀÒï0û+ _wandbruntime‚467  _runtime‚467é2Q—PüÊ _runtime‚ 467.731658237 /loss/FRt4/assignment_loss‚5.624078575221971 2loss/SRqqt1/assignment_loss‚3.3403156268887404 2loss/SRqqt3/assignment_loss‚2.7358135824472134 0loss/FBt3/assignment_loss‚0.3831725164570592 /loss/FRt2/detection_loss‚3.8652179212797257 1loss/SRqqt3/detection_loss‚1.8734435766122082 2loss/SRqqt2/assignment_loss‚3.0241655027657224 .loss/FRt3/detection_loss‚2.935230279820306 1loss/SRqqt4/assignment_loss‚3.265309702605009 0loss/SRqqt2/detection_loss‚2.373428865183054 1loss/SRqqt4/detection_loss‚2.2256053506571334 .loss/FBt1/detection_loss‚7.290218740338233 -loss/FBt3/detection_loss‚4.34161492017209 epoch‚0 /loss/FRt4/detection_loss‚3.5833197311423293 1loss/SRqqt1/detection_loss‚3.5812061539873845 _step‚47 /loss/FRt2/assignment_loss‚5.949474340393429 0loss/FBt1/assignment_loss‚0.5094231327728754 .loss/FRt1/detection_loss‚3.238999477366811 /loss/FBt2/detection_loss‚1.8863787489401984 %loss/total_loss‚84.75098074515077 trainer/global_step‚599 $ _timestamp‚1.788393694752048e+09 /loss/FRt1/assignment_loss‚5.613278280828417 0loss/FBt2/assignment_loss‚0.2990484539711996 /loss/FBt4/detection_loss‚10.777049743466907 /loss/FRt3/assignment_loss‚5.623074415751866 0loss/FBt4/assignment_loss‚0.4121131060808828 pXùTýÎ /loss/FRt3/assignment_loss‚5.623074415751866 0loss/FBt4/assignment_loss‚0.4121131060808828 _runtime‚ 467.731658237 /loss/FRt4/assignment_loss‚5.624078575221971 2loss/SRqqt1/assignment_loss‚3.3403156268887404 2loss/SRqqt3/assignment_loss‚2.7358135824472134 0loss/FBt3/assignment_loss‚0.3831725164570592 /loss/FRt2/detection_loss‚3.8652179212797257 1loss/SRqqt3/detection_loss‚1.8734435766122082 2loss/SRqqt2/assignment_loss‚3.0241655027657224 .loss/FRt3/detection_loss‚2.935230279820306 1loss/SRqqt4/assignment_loss‚3.265309702605009 0loss/SRqqt2/detection_loss‚2.373428865183054 1loss/SRqqt4/detection_loss‚2.2256053506571334 .loss/FBt1/detection_loss‚7.290218740338233 -loss/FBt3/detection_loss‚4.34161492017209 epoch‚0 /loss/FRt4/detection_loss‚3.5833197311423293 1loss/SRqqt1/detection_loss‚3.5812061539873845 _step‚47 /loss/FRt2/assignment_loss‚5.949474340393429 0loss/FBt1/assignment_loss‚0.5094231327728754 .loss/FRt1/detection_loss‚3.238999477366811 /loss/FBt2/detection_loss‚1.8863787489401984 %loss/total_loss‚84.75098074515077 trainer/global_step‚599 $ _timestamp‚1.788393694752048e+09 /loss/FRt1/assignment_loss‚5.613278280828417 0loss/FBt2/assignment_loss‚0.2990484539711996 /loss/FBt4/detection_loss‚10.777049743466907/Òð/Ÿ‘þ‚(:† ëñâÔÇ”Ý  gpu.0.gpu‚3.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.684' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5368 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‘p›¹'ÿ‚(:œ ëñâÔÁ´Ø  disk.sda1.in‚0 disk.sda2.out‚0 network.sent‚751755 cpu‚21.331531559609704 disk.dm-1.out‚ 478.9609375 network.recv‚221884) disk./.usagePercent‚15.61852121733098 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.sda3.in‚0 disk.nvme0c0n1.in‚0 disk.drbd1716.in‚0 disk.sda1.out‚0 disk.sda3.out‚ 478.9609375 disk.rbd1.in‚0$ memory_percent‚80.99255561828613( proc.memory.availableMB‚ 3114.1796875# proc.memory.rssMB‚ 2806.48828125 disk.rbd0.in‚0 disk.drbd2265.out‚0) proc.memory.percent‚17.12944507598877 proc.cpu.threads‚17% disk./.usageGB‚232.72725296020508# disk.nvme0c0n1.out‚ 19.408203125 disk.dm-0.out‚ 19.408203125 disk.drbd2080.out‚0! disk.nvme0n1.out‚ 19.408203125 disk.sda.in‚0 disk.dm-1.in‚0 disk.rbd1.out‚ 2.3271484375 disk.dm-0.in‚0 disk.drbd2266.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚0 disk.sda.out‚ 478.9609375 disk.drbd2265.in‚0Óûs´’€‚(:‡ úñâÔ湌  gpu.0.gpu‚63.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.684' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5368 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0KÊ—s(‚(: úñâÔ€ÂÐ% disk./.usageGB‚232.72725296020508" proc.memory.rssMB‚ 2892.8984375 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.sda.out‚ 498.59765625 disk.drbd2266.in‚0 network.sent‚800860" disk.nvme0c0n1.out‚ 19.87890625 disk.rbd0.out‚0$ memory_percent‚81.51581287384033 proc.cpu.threads‚17 disk.sda3.in‚0 disk.dm-0.out‚ 19.87890625 disk.sda2.out‚0 disk.drbd1716.out‚0 network.recv‚226794 disk.dm-1.in‚0 disk.dm-0.in‚0 disk.rbd0.in‚0 disk.rbd1.out‚ 2.4990234375 disk.sda3.out‚ 498.59765625 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.nvme0n1.in‚0) proc.memory.availableMB‚ 3028.44921875 cpu‚21.597578750061395) disk./.usagePercent‚15.61852121733098 disk.nvme0c0n1.in‚0 disk.sda2.in‚0 disk.sda.in‚0* proc.memory.percent‚17.656850814819336 disk.drbd2080.out‚0 disk.nvme0n1.out‚ 19.87890625 disk.drbd2265.out‚0 disk.sda1.out‚0 disk.dm-1.out‚ 498.59765625 disk.rbd1.in‚0 disk.drbd2266.out‚0§:Xñ;‚b6 jet/accuracy_6_of_7"trainer/global_step2:JPÙ=JƒbE "Purity/*FRt1SRqqt4FBt/event_purity"trainer/global_step2:JP~uåJ„bE "Purity/*FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP‹ØbÇN…bI &Purity/2FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP 6[êN†bI &Purity/2FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP; Z¿9‡b4 particle/accuracy"trainer/global_step2:JPÁˆ·HˆbC Purity/3FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPCš9¦J‰bE "Purity/1FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPóŒP'JŠbE "Purity/1FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP«8|N‹bI &Purity/4FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP¤rfV@Œb; particle/accuracy_4_of_4"trainer/global_step2:JP "G\;b6 jet/accuracy_4_of_6"trainer/global_step2:JPÇ:CÂ@Žb; particle/accuracy_2_of_6"trainer/global_step2:JPl‰RàNbI &Purity/0FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP3þ|-JbE "Purity/1FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¨E•J‘bE "Purity/1FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPqšÆJ’bE "Purity/3FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP¡îu‡N“bI &Purity/*FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPw®ÐÏH”bC Purity/2FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JPX0°N•bI &Purity/2FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP|‚!K@–b; particle/accuracy_3_of_7"trainer/global_step2:JP zäH—bC Purity/1FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP3‘²ÁH˜bC Purity/1FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP³’.J™bE "Purity/2FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÖzîhNšbI &Purity/4FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPg`$›J›bE "Purity/0FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPJDšiJœbE "Purity/0FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPÁ°ÊJbE "Purity/1FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPjZ¤òNžbI &Purity/3FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP×­“rJŸbE "Purity/1FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPD÷òÄ; b6 jet/accuracy_7_of_7"trainer/global_step2:JP’v†8J¡bE "Purity/1FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP) îN¢bI &Purity/*FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPNöp7H£bC Purity/*FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPôôœ(N¤bI &Purity/1FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP¾ÎpN¥bI &Purity/3FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP:Ç_¢N¦bI &Purity/3FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP–1bN§bI &Purity/3FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPÎÛä_J¨bE "Purity/0FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPtÀäH©bC Purity/*FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JPí×"ÔJªbE "Purity/3FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÙ´H«bC Purity/1FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JPÐV3yH¬bC Purity/3FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP¦ªßJ­bE "Purity/1FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¿)à©N®bI &Purity/*FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPu&wN¯bI &Purity/2FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP(lj¡N°bI &Purity/4FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPwÄ9;;±b6 jet/accuracy_2_of_4"trainer/global_step2:JPöUiJ²bE "Purity/3FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPŽFM J³bE "Purity/*FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP&]D²J´bE "Purity/*FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP°îHµbC Purity/2FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP¶G 8;¶b6 jet/accuracy_1_of_3"trainer/global_step2:JPb6™eH·bC Purity/2FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JP2ÙýJ¸bE "Purity/*FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPŸ°BÙJ¹bE "Purity/*FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP"Í(QJºbE "Purity/1FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPÁ„ÛH»bC Purity/1FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPæÕ9N¼bI &Purity/3FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPó9ZõN½bI &Purity/4FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP ]½0H¾bC Purity/1FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JP>ìƒrH¿bC Purity/2FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPxÆwHÀbC Purity/*FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPŸ¨ÝèJÁbE "Purity/1FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPñHlNÂbI &Purity/1FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP–4€^JÃbE "Purity/2FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP g7ÞJÄbE "Purity/1FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPŒ0HÅbC Purity/*FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JPŠ¥P­NÆbI &Purity/0FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP4ý!OJÇbE "Purity/1FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP—!Z­HÈbC Purity/1FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP/¹ºHÉbC Purity/2FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPwݨ/NÊbI &Purity/2FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPùfš‘HËbC Purity/3FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPXµÈNÌbI &Purity/3FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP—\@Íb; particle/accuracy_5_of_7"trainer/global_step2:JP™2ä—JÎbE "Purity/*FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¢*/•HÏbC Purity/1FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP‚ïÎHÐbC Purity/1FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JP µJÑbE "Purity/*FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPtDÊHÒbC Purity/4FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPªË@?JÓbE "Purity/4FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPÆk"NÔbI &Purity/*FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP¬è¦~JÕbE "Purity/2FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPØH¢NÖbI &Purity/3FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPÃVkþJ×bE "Purity/3FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP|ÿÆNØbI &Purity/4FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPCª-HÙbC Purity/*FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPpÁfNÚbI &Purity/*FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPa:;ÛHÛbC Purity/1FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JPX‡¿JÜbE "Purity/3FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP"KGîNÝbI &Purity/0FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPøŸ•öJÞbE "Purity/1FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPšË[±NßbI &Purity/3FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP|0)HàbC Purity/2FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPÄ‘j NábI &Purity/*FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPgéÈÿHâbC Purity/1FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP°ûhJãbE "Purity/2FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP§B›ÁHäbC Purity/3FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPWˆÄJåbE "Purity/0FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPn­ÅqJæbE "Purity/1FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JPΫÑ!HçbC Purity/*FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPäV}NèbI &Purity/1FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP.ż8NébI &Purity/1FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP ûwêNêbI &Purity/2FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPÂNëbI &Purity/0FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP*õÉ„NìbI &Purity/2FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPÿçœNíbI &Purity/4FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JPÒŸÓºJîbE "Purity/2FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP*é"~JïbE "Purity/*FRt1SRqqt3FBt/event_purity"trainer/global_step2:JPLšÊãJðbE "Purity/*FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP"¿†HñbC Purity/*FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP¢'dÑNòbI &Purity/*FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP7öXÚNóbI &Purity/*FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPe?oÁNôbI &Purity/0FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP-YŸNõbI &Purity/2FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPÊâÌNöbI &Purity/2FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPÀÌ.ÎN÷bI &Purity/4FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPÕüHøbC Purity/2FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP:Ü¥‚JùbE "Purity/1FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPßòƒÈJúbE "Purity/*FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP°qÙ«JûbE "Purity/1FRt3SRqqt2FBt/event_purity"trainer/global_step2:JP­ŠdHübC Purity/*FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPbçÿHHýbC Purity/1FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPþñ«NþbI &Purity/3FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPã.ÞhNÿbI &Purity/4FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPÁ1,šH€bC Purity/3FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPsi7JbE "Purity/2FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPžë× H‚bC Purity/2FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP×úJƒbE "Purity/3FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPH ] J„bE "Purity/3FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP)×Ä[;…b6 jet/accuracy_4_of_7"trainer/global_step2:JP?o!H†bC Purity/*FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPøÑtN‡bI &Purity/0FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP’¡emJˆbE "Purity/0FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP"ºë N‰bI &Purity/0FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPGݨ2NŠbI &Purity/0FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPù`N‹bI &Purity/1FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPbˆ@äJŒbE "Purity/3FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP,ŸsüNbI &Purity/*FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP`-žJŽbE "Purity/1FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPùw›NbI &Purity/2FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP\4;6NbI &Purity/3FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPÿž"vN‘bI &Purity/4FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP.ãTlN’bI &Purity/4FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP¦»7Ì@“b; particle/accuracy_2_of_2"trainer/global_step2:JPã–y-J”bE "Purity/*FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPS5 mJ•bE "Purity/*FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP(‚ÖJ–bE "Purity/2FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPi¹6IH—bC Purity/2FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPLªÉo;˜b6 jet/accuracy_1_of_4"trainer/global_step2:JP%@J™bE "Purity/4FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPeSëÆJšbE "Purity/4FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¹:.ÅJ›bE "Purity/*FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP[T‘"HœbC Purity/0FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP*Ù‹çHbC Purity/1FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP4@óŠHžbC Purity/2FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPâ¾JùNŸbI &Purity/2FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPÉùŒÃH bC Purity/3FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPmÌSSN¡bI &Purity/4FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPW\‘N¢bI &Purity/4FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPÄ%eH£bC Purity/*FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPc›ºÂN¤bI &Purity/*FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPkÒ/&N¥bI &Purity/3FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPé?4N¦bI &Purity/4FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP”8?N§bI &Purity/4FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPçJ¨bE "Purity/0FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPHœ…}H©bC Purity/3FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP°p¯ÚJªbE "Purity/*FRt3SRqqt2FBt/event_purity"trainer/global_step2:JP(Ã3ÃJ«bE "Purity/1FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPUŽˆ N¬bI &Purity/1FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP½'v ­bE "Purity/3FRt*S}¬g6Rqqt1FBt/SRqqt_purity"trainer/global_step2:JPÇÆ1iJ®bE "Purity/4FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPËîfH¯bC Purity/0FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPªêÃH°bC Purity/0FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPä+ mJ±bE "Purity/1FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP+ËÆÐJ²bE "Purity/2FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPW„™H³bC Purity/3FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPvTßN´bI &Purity/3FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP1£¾ÙNµbI &Purity/3FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPSÍÁ!J¶bE "Purity/0FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPå,דH·bC Purity/1FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPŒ¯‘@J¸bE "Purity/1FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP>Ç»J¹bE "Purity/3FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPÊÒŽNºbI &Purity/3FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP8@–‚H»bC Purity/3FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP¥nŒÔH¼bC Purity/*FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JP㊬“N½bI &Purity/*FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPM5eJ¾bE "Purity/*FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP[Ô^H¿bC Purity/2FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPßîÉHÀbC Purity/3FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP³/5ÕJÁbE "Purity/0FRt4SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP‘¥JÂbE "Purity/2FRt4SRqqt*FBt/event_purity"trainer/global_step2:JP•\*BJÃbE "Purity/0FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPN"n±NÄbI &Purity/0FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP>á,sHÅbC Purity/2FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPI=ÓËJÆbE "Purity/2FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPó úJÇbE "Purity/1FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP …ð‰HÈbC Purity/1FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPZ§â—HÉbC Purity/3FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPñésHÊbC Purity/*FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JP61(ÌHËbC Purity/*FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JP ³|wJÌbE "Purity/0FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPùb?NÍbI &Purity/2FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP…˜OŒ;Îb6 jet/accuracy_2_of_5"trainer/global_step2:JP¾$Ý<Ïb7 particle/sensitivity"trainer/global_step2:JP´Ã™DJÐbE "Purity/0FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP!¡NÑbI &Purity/*FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPSµíJÒbE "Purity/2FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP8scmNÓbI &Purity/3FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPÒ=„ú@Ôb; particle/accuracy_6_of_6"trainer/global_step2:JPÍ u‹HÕbC Purity/3FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP±ÁýßHÖbC Purity/0FRt2SRqqt3FBt/FBt_purity"trainer/global_step2:JP%‹Ç>H×bC Purity/1FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPh[”HØbC Purity/3FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP2¥ßtHÙbC Purity/4FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPA½\JÚbE "Purity/*FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÛâpNÛbI &Purity/*FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPD5œÐJÜbE "Purity/3FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP<¦ÇJÝbE "Purity/0FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP£ø.@Þb; particle/accuracy_4_of_6"trainer/global_step2:JPÍŠâöJßbE "Purity/*FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPþñôSHàbC Purity/*FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPZ;÷ÏNábI &Purity/3FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPZch/@âb; particle/accuracy_4_of_5"trainer/global_step2:JPuñÔ\@ãb; particle/accuracy_5_of_6"trainer/global_step2:JPˈ—JäbE "Purity/*FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPÿÝ»ŽHåbC Purity/0FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP`Õú_JæbE "Purity/2FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP€oCJçbE "Purity/2FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPõ ›:JèbE "Purity/3FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPdt±êNébI &Purity/0FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP¤çJêbE "Purity/3FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPî_‚ÂHëbC Purity/*FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JP囯HìbC Purity/2FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPɃ1HíbC Purity/3FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP^\gJîbE "Purity/3FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÑmJïbE "Purity/0FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¨÷èaHðbC Purity/1FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPñn•ŽJñbE "Purity/1FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP‚ˆ‚JòbE "Purity/2FRt*SRqqt1FBt/event_purity"trainer/global_step2:JPèìoeNóbI &Purity/2FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP«ü/ê8ôb3 particle/f_score"trainer/global_step2:JP“ržVHõbC Purity/0FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP¸{‹HöbC Purity/4FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPH…Y¯H÷bC Purity/2FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP…17ÐNøbI &Purity/3FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPek 7JùbE "Purity/*FRt4SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP[>4JúbE "Purity/4FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPXz4XHûbC Purity/*FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPK˨)NübI &Purity/0FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPFú’ãJýbE "Purity/3FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPí£¦€HþbC Purity/0FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JP-tÒ¡HÿbC Purity/2FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPÆ-“ûJ€bE "Purity/3FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPTVQƒJbE "Purity/2FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP”û{×N‚bI &Purity/1FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPÕ•:}NƒbI &Purity/4FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP{¨áäH„bC Purity/2FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JP„’-xJ…bE "Purity/0FRt1SRqqt3FBt/event_purity"trainer/global_step2:JP³ÆÖH†bC Purity/*FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPµ—“ÞH‡bC Purity/1FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPÈbU±NˆbI &Purity/1FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP¤Ôà™J‰bE "Purity/2FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP…HÖNŠbI &Purity/3FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPÕ8¢ŠJ‹bE "Purity/1FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPõÕÈ™HŒbC Purity/*FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPZÅ'HbC Purity/3FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPÞÔHŽbC Purity/*FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPí1ðÿJbE "Purity/2FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP0z-}@b; particle/accuracy_1_of_6"trainer/global_step2:JPL_ÎJ‘bE "Purity/0FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPÞ•xN’bI &Purity/0FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP®G‚H“bC Purity/1FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JP SKJ”bE "Purity/0FRt4SRqqt1FBt/event_purity"trainer/global_step2:JPn‡éH•bC Purity/2FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPö©ºeJ–bE "Purity/0FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP˨ü‰H—bC Purity/1FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JP¾ŒRJ˜bE "Purity/0FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP&¶ŠÑN™bI &Purity/0FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPªŒRNšbI &Purity/0FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP g>H›bC Purity/2FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPE­QHœbC Purity/2FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JP -RŠJbE "Purity/*FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPÇí×fNžbI &Purity/0FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP{>ºáJŸbE "Purity/1FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP˜·¾™J bE "Purity/1FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP?ÞÅ&;Ûb6 jet/accuracy_3_of_5"trainer/global_step2:JPÐf”—@Üb; particle/accuracy_1_of_3"trainer/global_step2:JPpá)`;Ýb6 jet/accuracy_2_of_6"trainer/global_step2:JPÞ"•µHÞbC Purity/3FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JP6Ä$JßbE "Purity/2FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP'üñ¾HàbC Purity/*FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP²JÍNábI &Purity/3FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JP«ø. JâbE "Purity/*FRt4SRqqt1FBt/event_purity"trainer/global_step2:JPb3zJãbE "Purity/3FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP×DzCNäbI &Purity/*FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPBªNåbI &Purity/2FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP”@‹OHæbC Purity/*FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPƒ÷f NçbI &Purity/*FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP xNèbI &Purity/0FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP=Dc1NébI &Purity/2FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP>ÅÍGêbB validation_average_jet_accuracy"trainer/global_step2:JP±8¢(;ëb6 loss/val_total_loss"trainer/global_step2:JP²¢é‹JìbE "Purity/1FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPŸ ½NíbI &Purity/4FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP*XaHîbC Purity/*FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPãPîóJïbE "Purity/3FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPF#Že;ðb6 jet/accuracy_3_of_7"trainer/global_step2:JP; o1JñbE "Purity/1FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP^)ÍŽNòbI &Purity/1FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPå>oXHóbC Purity/1FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPüvp HôbC Purity/3FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPAГ`HõbC Purity/3FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP„Ò HöbC Purity/*FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JP7ö/H÷bC Purity/4FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP *.JøbE "Purity/4FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPãT½JùbE "Purity/*FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP  áJúbE "Purity/*FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP‹[¨ HûbC Purity/*FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JP)H|ÐHübC Purity/3FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JP>–ûHýbC Purity/1FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JP6¹}òNþbI &Purity/*FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP#¡”JÿbE "Purity/1FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¹Î6„N€bI &Purity/1FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP¶jðºHbC Purity/2FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP¸ÿâŸJ‚bE "Purity/2FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPúÍZNƒbI &Purity/2FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPô>d¦@„b; particle/accuracy_1_of_1"trainer/global_step2:JP¿êHl@…b; particle/accuracy_2_of_3"trainer/global_step2:JP¶K–œH†bC Purity/*FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPÏš!J‡bE "Purity/*FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPÝKóÔHˆbC Purity/1FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JP ¸BN‰bI &Purity/3FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPç…¯NŠbI &Purity/4FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPÈþ;‹b6 jet/accuracy_2_of_2"trainer/global_step2:JP‘ê#@Œb; particle/accuracy_1_of_2"trainer/global_step2:JPË~¹p<b7 particle/specificity"trainer/global_step2:JPÏ~ONŽbI &Purity/0FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP»ƒJbE "Purity/1FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPÙÜw¦HbC Purity/2FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP½xH‘bC Purity/2FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPF°ïðN’bI &Purity/4FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP±p8/J“bE "Purity/2FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPt)F`H”bC Purity/2FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPÕ¦îJ•bE "Purity/3FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¶Ç…H–bC Purity/*FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP–ÔâJ—bE "Purity/2FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP0è‘…;˜b6 jet/accuracy_1_of_2"trainer/global_step2:JP®X@™b; particle/accuracy_3_of_3"trainer/global_step2:JP 8èJšbE "Purity/0FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPže—âJ›bE "Purity/3FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPê¨cHœbC Purity/3FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPS׆JbE "Purity/*FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPí˜YNžbI &Purity/1FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPÛ»à;HŸbC Purity/1FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JP˜lH bC Purity/3FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JPe[0"N¡bI &Purity/4FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPNV!û@¢b; particle/accuracy_5_of_5"trainer/global_step2:JP ÷ßH£bC Purity/0FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JPåEx¤J¤bE "Purity/*FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPËPÆN¥bI &Purity/0FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPzR¡ˆN¦bI &Purity/2FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPѸW3J§bE "Purity/3FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP u`rN¨bI &Purity/4FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP´Ÿ´¡;©b6 jet/accuracy_2_of_3"trainer/global_step2:JPìdìK@ªb; particle/accuracy_3_of_4"trainer/global_step2:JPJ£àïH«bC Purity/*FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP°eŠJ¬bE "Purity/2FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPÎôH­bC Purity/2FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JP牨N®bI &Purity/3FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP¼àÅ…;¯b6 jet/accuracy_1_of_7"trainer/global_step2:JPfƒ¹Ë@°b; particle/accuracy_6_of_7"trainer/global_step2:JP¢õ‹ H±bC Purity/*FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JPÃ:NôN²bI &Purity/0FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP×ìeJ³bE "Purity/*FRt0SRqqt4FBt/event_purity"trainer/global_step2:JP(jЭH´bC Purity/1FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPŽ¯Ã¹JµbE "Purity/4FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPô{ÒðN¶bI &Purity/*FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPFó½®J·bE "Purity/0FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPÿ–N¸bI &Purity/0FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPÂÇ\N¹bI &Purity/1FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP*BÇNºbI &Purity/1FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP½cœN»bI &Purity/1FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP˜’âH¼bC Purity/2FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPã²[ÑH½bC Purity/4FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPPP¸gH¾bC Purity/*FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP.·Ï@N¿bI &Purity/*FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP¹uÌNÀbI &Purity/2FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP¤°JÁbE "Purity/1FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPð]zRJÂbE "Purity/4FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP÷q³'JÃbE "Purity/*FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPðàæHÄbC Purity/0FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPŠÄí~NÅbI &Purity/0FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP{Up£JÆbE "Purity/1FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPH`½NÇbI &Purity/2FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP©ÎÞÎJÈbE "Purity/3FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPH áJÉbE "Purity/1FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPA;zü@Êb; particle/accuracy_7_of_7"trainer/global_step2:JP'ßíNËbI &Purity/1FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPájxxNÌbI &Purity/*FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPô[ÛHÍbC Purity/0FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPÐPÞ¿HÎbC Purity/0FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP’ª±6HÏbC Purity/1FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPXNbÐNÐbI &Purity/1FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP ÂHÑbC Purity/1FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPHÙ­úJÒbE "Purity/2FRt*SRqqt0FBt/event_purity"trainer/global_step2:JP¶K §HÓbC Purity/*FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPMûINÔbI &Purity/0FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPs}ZNÕbI &Purity/1FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPÎ[†NÖbI &Purity/3FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP ëßN×bI &Purity/4FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPœ¼G@Øb; particle/accuracy_1_of_4"trainer/global_step2:JP? º;Ùb6 jet/accuracy_5_of_6"trainer/global_step2:JPÕ9ÔûJÚbE "Purity/*FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPâ\^tJÛbE "Purity/*FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPaá…NÜbI &Purity/*FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPQÕVNÝbI &Purity/1FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP0ºúNÞbI &Purity/1FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPwÃ^µNßbI &Purity/4FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPA"NàbI &Purity/4FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPù,HábC Purity/*FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPœB†JâbE "Purity/*FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP¡pœ„HãbC Purity/0FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP0/y•HäbC Purity/1FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPÙÔ•HåbC Purity/2FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JP¦`JæbE "Purity/2FRt3SRqqt1FBt/event_purity"trainer/global_step2:JP)S²IHçbC Purity/1FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JPÏœ‰yJèbE "Purity/*FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPç&é°NébI &Purity/0FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPB´8{NêbI &Purity/1FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP Õ\×JëbE "Purity/1FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP€×K-HìbC Purity/2FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP/}Ú‘HíbC Purity/4FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPX €JîbE "Purity/4FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP•£JïbE "Purity/4FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP°,IJðbE "Purity/*FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP4¤ ™JñbE "Purity/*FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPBFU1JòbE "Purity/0FRt*SRqqt4FBt/event_purity"trainer/global_step2:JP8×ØìJóbE "Purity/3FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPMKí8JôbE "Purity/0FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPâÔj>HõbC Purity/*FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPü†HöbC Purity/*FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP`&o,N÷bI &Purity/*FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPXG2;JøbE "Purity/0FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPu‘y”NùbI &Purity/1FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPoAtJúbE "Purity/2FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP¡B.ÈNûbI &Purity/2FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP;éõ½NübI &Purity/4FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP"l}JýbE "Purity/1FRt0SRqqt3FBt/event_purity"trainer/global_step2:JP«ªEºJþbE "Purity/3FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPTï¸ÉJÿbE "Purity/3FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPü€9¿J€bE "Purity/4FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPœÜWoHbC Purity/*FRt2SRqqt3FBt/FBt_purity"trainer/global_step2:JP®õßòJ‚bE "Purity/2FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPã?|¯NƒbI &Purity/2FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPÇÇ]N„bI &Purity/3FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPe­÷`@…b; particle/accuracy_1_of_5"trainer/global_step2:JPÌñŠN†bI &Purity/*FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP€¿‡^N‡bI &Purity/*FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP¹^ÑHˆbC Purity/*FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP¢ª­J‰bE "Purity/0FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPÏ°Ü×NŠbI &Purity/1FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPp,|J‹bE "Purity/*FRt0SRqqt3FBt/event_purity"trainer/global_step2:JP ?‚tNŒbI &Purity/1FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPÚôD¬NbI &Purity/2FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPv¾NŽbI &Purity/4FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPt‰6|JbE "Purity/3FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¢{IJbE "Purity/1FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPDÚ[N‘bI &Purity/0FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP…(RPN’bI &Purity/1FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP¬|RON“bI &Purity/1FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP*óŒ”J”bE "Purity/0FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPtG¦)H•bC Purity/3FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JP¢¥Ã—N–bI &Purity/*FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPÞ…þ@N—bI &Purity/*FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP`iæÒH˜bC Purity/0FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP«wŸyHÇbC Purity/3FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JP’ÃNNÈbI &Purity/0FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPÝèÁGNÉbI &Purity/0FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP/'ÓNÊbI &Purity/1FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPÔc²ÁNËbI &Purity/1FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPŽ‡^@Ìb; particle/accuracy_3_of_6"trainer/global_step2:JPÅ'4JÍbE "Purity/3FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPû£›òJÎbE "Purity/*FRt2SRqqt3FBt/event_purity"trainer/global_step2:JP‚Žk$HÏbC Purity/*FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPWúQJÐbE "Purity/*FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP’òþNÑbI &Purity/*FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPa;1HÒbC Purity/0FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPÒúÝ|JÓbE "Purity/1FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP³…»JÔbE "Purity/1FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÛ(†JÕbE "Purity/3FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPëV-NÖbI &Purity/4FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP`^ìH×bC Purity/*FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPnБ¥JØbE "Purity/*FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP•¥¿aNÙbI &Purity/1FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP]{dÈ;Úb6 jet/accuracy_4_of_5"trainer/global_step2:JP à‚sJÛbE "Purity/*FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPA„]aHÜbC Purity/*FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JP=¦ýlNÝbI &Purity/*FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPEïNÞbI &Purity/1FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPn=HßbC Purity/1FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JP YHàbC Purity/2FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JP,-\%JábE "Purity/2FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP"õ7NâbI &Purity/3FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPµ 0¸JãbE "Purity/0FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPwIHäbC Purity/*FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP_Ü3 JåbE "Purity/0FRt1SRqqt2FBt/event_purity"trainer/global_step2:JP¡j0HæbC Purity/1FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPŠOT›JçbE "Purity/0FRt2SRqqt3FBt/event_purity"trainer/global_step2:JPN9½NèbI &Purity/*FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPºAÛÆJébE "Purity/0FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPA'_ÚHêbC Purity/1FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPIýÕ8JëbE "Purity/2FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP½.HìbC Purity/0FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPXžvHíbC Purity/4FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP¶ÚßHîbC Purity/*FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP;ÍÃÜNïbI &Purity/*FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPHüÊHðbC Purity/*FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPïaÏËJñbE "Purity/0FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP§*J…NòbI &Purity/1FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPÎ]±NóbI &Purity/*FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPÄ ‘SNôbI &Purity/*FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPÐ"«èJõbE "Purity/0FRt0SRqqt3FBt/event_purity"trainer/global_step2:JP½–3ôJöbE "Purity/1FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¸HÿbC Purity/*FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JP„oŽH€ bC Purity/3FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPÎÆÒa; b6 jet/accuracy_4_of_4"trainer/global_step2:JPNk•vJ‚ bE "Purity/1FRt*SRqqt4FBt/event_purity"trainer/global_step2:JPÛߺHƒ bC Purity/1FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JP ÿ}PJ„ bE "Purity/2FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPäåx™H… bC Purity/2FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP¦ªëH† bC Purity/*FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPlêòJ‡ bE "Purity/0FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPAñcNˆ bI &Purity/2FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP 38—;‰ b6 jet/accuracy_3_of_3"trainer/global_step2:JP<XuJŠ bE "Purity/*FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPXÏöáJ‹ bE "Purity/*FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÀŸUðNŒ bI &Purity/0FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPW¾‹áN bI &Purity/0FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP”"ƒ!HŽ bC Purity/2FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPžÚJ bE "Purity/3FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPËœÉN bI &Purity/3FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPoJ‘ bE "Purity/*FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPâ^RH’ bC Purity/*FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPßF³HN“ bI &Purity/0FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP¤×ùJ” bE "Purity/1FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP¤]‹®N• bI &Purity/2FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPàƒxß;– b6 jet/accuracy_1_of_6"trainer/global_step2:JP^WõH— bC Purity/1FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JP &Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 8"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.054572943598032 & Purity/*FRt1SRqqt4FBt/FBt_purity‚1 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.09259258955717087 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0 trainer/global_step‚603 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_4‚0.11321017891168594 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.1798250526189804 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.1417056918144226 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 *jet/accuracy_1_of_3‚0.5703610181808472 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.1428571492433548 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.13549858331680298 ("Purity/1FRt0SRqqt2FBt/event_purity‚0 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.05621305853128433 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.004018520470708609 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.13709399104118347 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9987593293190002 9 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.008888889104127884 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.08029128611087799 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9906516075134277 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ("Purity/2FRt*SRqqt*FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.0692528784275055 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.02688172087073326 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 9 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.007575757801532745 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 _step‚48 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.06303240358829498 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.11576555669307709 9 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.014216285198926926 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 9 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.029644953086972237 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.050931792706251144 & Purity/1FRt0SRqqt3FBt/FRt_purity‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FRt_purity‚0 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.08046095818281174 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.05567050352692604 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9959248304367065 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 <"Purity/0FRt2SRqqt0FBt/event_purity‚0.0013440860202535987 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.0476190485060215 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.04288274794816971 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.015625 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.07976190745830536 *jet/accuracy_1_of_4‚0.6212920546531677 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 ;"Purity/*FRt1SRqqt1FBt/event_purity‚0.019942324608564377 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.014550264924764633 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 9 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.044238682836294174 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.03635188937187195 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0714285746216774 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.09781627357006073 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.1388888955116272 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9977879524230957 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 ;"Purity/1FRt*SRqqt1FBt/event_purity‚0.003421488218009472 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.06853967905044556 & Purity/3FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9687500596046448 ;"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.042217105627059937 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.0450839027762413 : Purity/2FRt0SRqqt2FBt/FRt_purity‚0.0059523810632526875 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.12778781354427338 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.046893153339624405 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0 & Purity/1FRt*SRqqt3FBt/FRt_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12560386955738068 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9761905074119568 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.07170294225215912 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 )jet/accuracy_2_of_5‚0.131269171833992 +particle/sensitivity‚0.3466263711452484 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.09615384787321091 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.06635421514511108 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_6_of_6‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9960122108459473 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.06002645567059517 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.10449736565351486 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.04722222313284874 particle/accuracy_4_of_6‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9992495179176331 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_6‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.03679767996072769 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9991706013679504 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.12719298899173737 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 & Purity/3FRt0SRqqt1FBt/FRt_purity‚0 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.07241658121347427 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt1SRqqt1FBt/event_purity‚0 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 'particle/f_score‚0.3206522464752197 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 % _timestamp‚1.7883937339671233e+09 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.028671061620116234 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.041015878319740295 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9911994934082031 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.054564762860536575 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.08570937067270279 particle/accuracy_1_of_6‚1 <"Purity/0FRt2SRqqt*FBt/event_purity‚0.0011123470030725002 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.04363444447517395 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/2FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.5955221652984619 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.05613718926906586 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.00830500852316618 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/*FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.20000000298023224 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_2_of_7‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.03407558426260948 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.02023867331445217 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.053298138082027435 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_5‚0 particle/accuracy_2_of_4‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.10087718814611435 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.02400055341422558 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.0714285746216774 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 ;"Purity/*FRt*SRqqt0FBt/event_purity‚0.021792612969875336 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.15816397964954376 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.11193913221359253 9 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.014444444328546524 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.0615939199924469 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.03938441723585129 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt0SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.05556558817625046 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.03181818500161171 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ("Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0 jet/accuracy_3_of_6‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 -jet/accuracy_3_of_5‚0.0021505376789718866 particle/accuracy_1_of_3‚1 +jet/accuracy_2_of_6‚0.12280702590942383 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.037887196987867355 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.0555555559694767 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9922905564308167 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 7validation_average_jet_accuracy‚0.24120885133743286 )loss/val_total_loss‚82.74449264317953 ;"Purity/1FRt0SRqqt1FBt/event_purity‚0.006777414586395025 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9968394637107849 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.04732036218047142 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.13383488357067108 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.09599719196557999 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.03119676187634468 6 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.118055559694767 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.0654858723282814 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.03375576436519623 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.11132872849702835 @&Purity/2F–ŒŒ<Rt4SRqqt*FBt/event_proportion‚0.0003150201519019902 particle/accuracy_1_of_1‚1 particle/accuracy_2_of_3‚1 9 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.017158495262265205 <"Purity/*FRt2SRqqt*FBt/event_purity‚0.0005376344197429717 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.03361911326646805 particle/accuracy_1_of_2‚1 +particle/specificity‚0.8491423726081848 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.04199738800525665 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.06666667014360428 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 :"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.12719298899173737 *jet/accuracy_1_of_2‚0.4687849283218384 particle/accuracy_3_of_3‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.007956358604133129 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.06130952388048172 9 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.025641025975346565 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10459770262241364 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.07692307978868484 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.07423419505357742 particle/accuracy_3_of_4‚1 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚0 particle/accuracy_6_of_7‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/1FRt2SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.18518517911434174 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 & Purity/2FRt2SRqqt2FBt/FRt_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.027790553867816925 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.0973832905292511 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9928209781646729 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/1FRt1SRqqt0FBt/event_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ("Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.995948851108551 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.03304050490260124 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.056177277117967606 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9377394914627075 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 jet/accuracy_5_of_6‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.0476190485060215 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.5933672189712524 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9929797649383545 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.04494539275765419 & Purity/2FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.004032257944345474 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.07407407462596893 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.06513115018606186 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3265730142593384 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.06666667014360428 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9981533885002136 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.05325140804052353 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.05892243981361389 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_5‚1 _runtime‚ 506.949212965 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9305556416511536 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8033396005630493 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.11428571492433548 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.0625 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9331276416778564 & Purity/1FRt3SRqqt1FBt/FRt_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.12962403893470764 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.018485812470316887 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.07699965685606003 particle/accuracy_3_of_5‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.07473810762166977 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.993830144405365 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.26677456498146057 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.03525345399975777 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.22177837789058685 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.04821428656578064 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 9 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.028248107060790062 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.09921915829181671 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 epoch‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.2955649793148041 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.03811594098806381 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.01308528520166874 9 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.038929540663957596 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.7106081247329712 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.0476190485060215 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 & Purity/*FRt0SRqqt3FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.07457117736339569 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9940677881240845 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.08141704648733139 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.11578339338302612 ;"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.032608695328235626 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9949066638946533 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.03173304721713066 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 jet/accuracy_4_of_5‚0 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.015625 & Purity/*FRt2SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.029569895938038826 9 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.035087719559669495 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.14702381193637848 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.012316715903580189 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.006666666828095913 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994135499000549 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.988095223903656 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.966666579246521 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 9 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.052829448133707047 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.07170294225215912 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.20000000298023224 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.0704926922917366 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.932222306728363 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.08413954824209213 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_1‚0.2561053931713104 ("Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 & Purity/*FRt*SRqqt3FBt/FRt_purity‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 -jet/accuracy_4_of_4‚0.0011123470030725002 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 6 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.054847601801157 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.06888888776302338 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 -jet/accuracy_3_of_3‚0.0034770776983350515 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.27212512493133545 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.08928251266479492 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.06000422313809395 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.013037962839007378 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/1FRt*SRqqt2FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.7105262875556946 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.06613586843013763 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 <"Purity/*FRt2SRqqt0FBt/event_purity‚0.0006325111025944352 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.06021476536989212 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.9166666865348816 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.06634415686130524 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.03115079179406166 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.06175501272082329 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.12649275362491608 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_7‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt*FBt/FRt_purity‚0LØLffCÓ £  _step‚48 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.06303240358829498 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.11576555669307709 9 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.014216285198926926 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 9 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.029644953086972237 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.050931792706251144 & Purity/1FRt0SRqqt3FBt/FRt_purity‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FRt_purity‚0 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.08046095818281174 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.05567050352692604 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9959248304367065 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 <"Purity/0FRt2SRqqt0FBt/event_purity‚0.0013440860202535987 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.0476190485060215 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.04288274794816971 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.015625 ("Purity/*FRt2SRqqt1FBt/event_purity‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.07976190745830536 *jet/accuracy_1_of_4‚0.6212920546531677 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 ;"Purity/*FRt1SRqqt1FBt/event_purity‚0.019942324608564377 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.014550264924764633 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 9 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.044238682836294174 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.03635188937187195 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0714285746216774 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.09781627357006073 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.1388888955116272 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9977879524230957 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 ;"Purity/1FRt*SRqqt1FBt/event_purity‚0.003421488218009472 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.06853967905044556 & Purity/3FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9687500596046448 ;"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.042217105627059937 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.0450839027762413 : Purity/2FRt0SRqqt2FBt/FRt_purity‚0.0059523810632526875 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.12778781354427338 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.046893153339624405 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0 & Purity/1FRt*SRqqt3FBt/FRt_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.12560386955738068 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9761905074119568 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.07170294225215912 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 )jet/accuracy_2_of_5‚0.131269171833992 +particle/sensitivity‚0.3466263711452484 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.09615384787321091 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.06635421514511108 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_6_of_6‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9960122108459473 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.06002645567059517 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.10449736565351486 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.04722222313284874 particle/accuracy_4_of_6‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9992495179176331 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_6‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.03679767996072769 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9991706013679504 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.12719298899173737 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 & Purity/3FRt0SRqqt1FBt/FRt_purity‚0 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.07241658121347427 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt1SRqqt1FBt/event_purity‚0 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 'particle/f_score‚0.3206522464752197 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 % _timestamp‚1.7883937339671233e+09 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.028671061620116234 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.041015878319740295 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9911994934082031 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.054564762860536575 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.08570937067270279 particle/accuracy_1_of_6‚1 <"Purity/0FRt2SRqqt*FBt/event_purity‚0.0011123470030725002 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.04363444447517395 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/2FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.5955221652984619 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.05613718926906586 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.00830500852316618 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/*FRt4SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.20000000298023224 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_2_of_7‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.03407558426260948 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.02023867331445217 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.053298138082027435 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_5‚0 particle/accuracy_2_of_4‚1 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.10087718814611435 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.02400055341422558 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.0714285746216774 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 ;"Purity/*FRt*SRqqt0FBt/event_purity‚0.021792612969875336 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.15816397964954376 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.11193913221359253 9 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.014444444328546524 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.0615939199924469 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.03938441723585129 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt0SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.05556558817625046 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.03181818500161171 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ("Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0 jet/accuracy_3_of_6‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 -jet/accuracy_3_of_5‚0.0021505376789718866 particle/accuracy_1_of_3‚1 +jet/accuracy_2_of_6‚0.12280702590942383 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.037887196987867355 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.0555555559694767 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9922905564308167 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 7validation_average_jet_accuracy‚0.24120885133743286 )loss/val_total_loss‚82.74449264317953 ;"Purity/1FRt0SRqqt1FBt/event_purity‚0.006777414586395025 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_3_of_7‚0 ("Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9968394637107849 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.04732036218047142 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.13383488357067108 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.09599719196557999 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.03119676187634468 6 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.118055559694767 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.0654858723282814 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.03375576436519623 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.11132872849702835 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 particle/accuracy_1_of_1‚1 particle/accuracy_2_of_3‚1 9 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.017158495262265205 <"Purity/*FRt2SRqqt*FBt/event_purity‚0.0005376344197429717 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.03361911326646805 particle/accuracy_1_of_2‚1 +particle/specificity‚0.8491423726081848 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.04199738800525665 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.06666667014360428 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.992519736289978 :"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.12719298899173737 *jet/accuracy_1_of_2‚0.4687849283218384 particle/accuracy_3_of_3‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.007956358604133129 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.06130952388048172 9 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.025641025975346565 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10459770262241364 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.07692307978868484 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.07423419505357742 particle/accuracy_3_of_4‚1 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚0 particle/accuracy_6_of_7‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/1FRt2SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.18518517911434174 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Pur.aû”ÄLity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 & Purity/2FRt2SRqqt2FBt/FRt_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.027790553867816925 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.0973832905292511 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9928209781646729 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/1FRt1SRqqt0FBt/event_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ("Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 particle/accuracy_7_of_7‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 6 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.995948851108551 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.03304050490260124 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 9 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.056177277117967606 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9377394914627075 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 jet/accuracy_5_of_6‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.0476190485060215 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.5933672189712524 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9929797649383545 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.04494539275765419 & Purity/2FRt4SRqqt0FBt/FRt_purity‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.004032257944345474 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.07407407462596893 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.06513115018606186 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3265730142593384 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.06666667014360428 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9981533885002136 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.05325140804052353 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.05892243981361389 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_5‚1 _runtime‚ 506.949212965 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9305556416511536 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8033396005630493 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.11428571492433548 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.0625 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9331276416778564 & Purity/1FRt3SRqqt1FBt/FRt_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.12962403893470764 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 9 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.018485812470316887 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.07699965685606003 particle/accuracy_3_of_5‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.07473810762166977 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.993830144405365 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.26677456498146057 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.03525345399975777 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_7‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.22177837789058685 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.04821428656578064 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 9 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.028248107060790062 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.09921915829181671 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 epoch‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.2955649793148041 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.03811594098806381 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.01308528520166874 9 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.038929540663957596 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.7106081247329712 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.0476190485060215 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 & Purity/*FRt0SRqqt3FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.07457117736339569 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9940677881240845 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.08141704648733139 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.11578339338302612 ;"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.032608695328235626 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9949066638946533 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.03173304721713066 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 jet/accuracy_4_of_5‚0 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.015625 & Purity/*FRt2SRqqt2FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 9 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.029569895938038826 9 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.035087719559669495 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.14702381193637848 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.012316715903580189 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.006666666828095913 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994135499000549 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt2SRqqt1FBt/event_purity‚0 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.988095223903656 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.966666579246521 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 9 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.052829448133707047 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.07170294225215912 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.20000000298023224 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.0704926922917366 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.932222306728363 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.08413954824209213 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_1‚0.2561053931713104 ("Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 & Purity/*FRt*SRqqt3FBt/FRt_purity‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 -jet/accuracy_4_of_4‚0.0011123470030725002 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 6 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.054847601801157 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.06888888776302338 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 -jet/accuracy_3_of_3‚0.0034770776983350515 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.27212512493133545 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.08928251266479492 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.06000422313809395 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.013037962839007378 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/1FRt*SRqqt2FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.7105262875556946 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.06613586843013763 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 <"Purity/*FRt2SRqqt0FBt/event_purity‚0.0006325111025944352 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.06021476536989212 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.9166666865348816 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.06634415686130524 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.03115079179406166 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.06175501272082329 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.12649275362491608 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_7‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.07892666757106781 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9939814805984497 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.13782641291618347 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_2_of_5‚1 & Purity/1FRt1SRqqt3FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.03933150693774223 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ;"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.061666667461395264 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9929396510124207 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.00960112176835537 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.07371428608894348 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.04027777910232544 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.009569029323756695 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 jet/accuracy_6_of_6‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.0714285746216774 <"Purity/1FRt*SRqqt*FBt/event_purity‚0.0037699600216001272 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 (particle/accuracy‚0.7706706523895264 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.07791177183389664 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 jet/accuracy_4_of_6‚0 particle/accuracy_2_of_6‚1 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.10068028420209885 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_7‚1 9 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.029186300933361053 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.00960112176835537 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.08683418482542038 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.1458333283662796 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 ("Purity/1FRt1SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.0833333358168602 jet/accuracy_7_of_7‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 9 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.011111111380159855 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 8"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.054572943598032 & Purity/*FRt1SRqqt4FBt/FBt_purity‚1 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.09259258955717087 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0 trainer/global_step‚603 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_4‚0.11321017891168594 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.1798250526189804 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.1417056918144226 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 *jet/accuracy_1_of_3‚0.5703610181808472 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.1428571492433548 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.13549858331680298 ("Purity/1FRt0SRqqt2FBt/event_purity‚0 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.05621305853128433 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.004018520470708609 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.13709399104118347 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9987593293190002 9 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.008888889104127884 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.08029128611087799 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9906516075134277 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ("Purity/2FRt*SRqqt*FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.0692528784275055 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.02688172087073326 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 9 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.007575757801532745 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚00ݸ+k‘Ô ‚(:† ‰òâÔûΰ  gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚35.0 gpu.0.powerWatts‚164.825' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.965 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0é 2ê+Õ ‚(:  ‰òâÔŒº¤  disk.rbd0.out‚0 disk.dm-1.in‚0 disk.dm-0.out‚20.21435546875 disk.sda.out‚ 512.953125$ memory_percent‚83.11541080474854 disk.drbd2265.out‚0 disk.rbd1.in‚0) proc.memory.percent‚17.79916286468506 cpu‚21.817706202649102 network.sent‚806364* disk./.usagePercent‚15.634185294321007 disk.sda3.in‚0 disk.drbd1716.out‚0) proc.memory.availableMB‚ 2766.37109375 disk.sda3.out‚ 512.953125 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0# proc.memory.rssMB‚ 2916.21484375 proc.cpu.threads‚17 disk.sda1.in‚0 disk.sda2.in‚0# disk.nvme0n1.out‚20.21435546875 disk.nvme0n1.in‚0 network.recv‚230503$ disk./.usageGB‚232.9606590270996 disk.drbd2265.in‚0 disk.nvme0c0n1.in‚0 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.sda.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 512.953125 disk.rbd1.out‚ 2.4990234375% disk.nvme0c0n1.out‚20.21435546875 disk.dm-0.in‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0k‚†œÖ ‚(:‘ ˜òâԜѽ  gpu.0.gpu‚50.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.782302856445312* gpu.0.memoryAllocatedBytes‚ 10120921088 gpu.0.temp‚36.0 gpu.0.powerWatts‚180.747' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚36.14940000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ç“82× ‚(:§ ˜òâÔ„Íê  network.recv‚237690 disk.drbd2265.in‚0 disk.sda3.in‚0& disk.nvme0c0n1.out‚259.22119140625 disk.drbd1716.in‚0 disk.nvme0n1.in‚0 disk.sda.out‚ 526.81640625 disk.drbd2265.out‚0 disk.dm-1.out‚ 526.81640625 disk.drbd1716.out‚0) proc.memory.availableMB‚ 2765.27734375 proc.cpu.threads‚17 disk.rbd0.out‚0 disk.rbd1.out‚ 2.5615234375! disk.dm-0.out‚259.22119140625 network.sent‚908426$ disk.nvme0n1.out‚259.22119140625 disk.drbd2266.in‚0 disk.sda3.out‚ 526.81640625 disk.rbd1.in‚0 disk.sda2.in‚0 disk.sda2.out‚0* disk./.usagePercent‚15.634120780422379 disk.drbd2080.out‚0$ memory_percent‚83.12208652496338 cpu‚21.9068137036369% disk./.usageGB‚232.95969772338867 disk.rbd0.in‚0 disk.sda.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.nvme0c0n1.in‚0! proc.memory.rssMB‚ 2916.890625 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.dm-0.in‚0 disk.drbd2080.in‚0* proc.memory.percent‚17.803287506103516Z£} Ø ‚(:’ §òâÔó÷¸  gpu.0.gpu‚46.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.223' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.244600000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0è| /'Ù ‚(:œ §òâÔ뾃  disk.rbd0.out‚0 network.recv‚242409 disk.drbd2266.out‚0 disk.sda2.in‚0 disk.sda.out‚542.625 disk.dm-1.out‚542.625 disk.sda3.out‚542.625& disk.nvme0c0n1.out‚259.37353515625 disk.sda2.out‚0 disk.drbd1716.in‚0 cpu‚21.991304498969836 network.sent‚914712* disk./.usagePercent‚15.634120780422379 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0# proc.memory.rssMB‚ 2916.89453125% disk./.usageGB‚232.95969772338867 disk.rbd0.in‚0$ memory_percent‚83.12056064605713 proc.cpu.threads‚17 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.drbd2266.in‚0 disk.nvme0c0n1.in‚0! disk.dm-0.out‚259.37353515625) proc.memory.availableMB‚ 2765.52734375 disk.nvme0n1.in‚0 disk.drbd2265.in‚0 disk.dm-0.in‚0 disk.drbd2080.out‚0* proc.memory.percent‚17.803311347961426$ disk.nvme0n1.out‚259.37353515625 disk.sda.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 2.7333984375ôJ8û0Ú + _wandbruntime‚543  _runtime‚543’[SÆÛ À ' lr-AdamW/pg1‚0.00012034268248414036 ' lr-AdamW/pg2‚0.00012034268248414036 trainer/global_step‚649 % _timestamp‚1.7883937709691753e+09 _runtime‚ 543.948989938 _step‚49ûë¦EÊÜ Ä ' lr-AdamW/pg1‚0.00012034268248414036 ' lr-AdamW/pg2‚0.00012034268248414036 trainer/global_step‚649 % _timestamp‚1.7883937709691753e+09 _runtime‚ 543.948989938 _step‚491Ž|&>0Ý + _wandbruntime‚543  _runtime‚543D.iSªÞ «O WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 _runtime‚ 543.953264626 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9103358 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚480 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3187199 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚34842317252608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8672919 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚36375942730752 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚35898921960448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 trainer/global_step‚649 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 477201694720 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9103360 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚11964988 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35898904921088 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5485720 D>DeviceStatsMonitor.on_train_batch_start/oversize_ {ïásegments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚35316998653683 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚36376123655168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 495622690816 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36810401268736 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚36315014986752 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚36314990083072 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9103360R™Œxµ'ß ¯O MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚35898904921088 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 _step‚50 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚9103358 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚34842300213248 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11984512 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚483 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5485717 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 474521328767 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 % _timestamp‚1.7883937709703543e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚36810637677568 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 474681401075 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚36376123655168 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚36375942730752 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚35898921960448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 trainer/global_step‚649 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 477201694720 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9103360 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚11964988 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35898904921088 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5485720 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚35316998653683 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚36376123655168 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 495622690816 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36810401268736 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚36315014986752 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚36314990083072 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9103360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 _runtime‚ 543.953264626 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9103358 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚480 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3187199 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚34842317252608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8672919 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 I8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚34894234976256 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚34894217936896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚36430043399680 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚36430224337408 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5493776 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚649 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11982980 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚35952294843392 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 477748556288 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 496365099520 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9116854 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35952311882752 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 477748556288 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚458 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3191393 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚21119386 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 236395520 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12002530 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21119386 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 477912454656 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 477912454656 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11982980 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚35369446225123 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3191848 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 496153607680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚21099834 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚36369016815616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211491840 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8685169 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚36865381915136 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9116856 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 496153607680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚21099834 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚36369016815616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211491840 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8685169 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚36865381915136 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9116856 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚34894234976256 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚34894217936896 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚36430043399680 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚36430224337408 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5493776 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚649 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11982980 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚35952294843392 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 477748556288 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 496365099520 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9116854 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35952311882752 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 477748556288 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚458 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3191393 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚21119386 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 236395520 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12002530 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21119386 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 477912454656 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 477912454656 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11982980 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚35369446225123 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3191848 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12887464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12867940 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39081184929792 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚38568401036288 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚37948259981091 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9778158 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚38568418075648 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚38568401036288 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚39015106429952 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚38568418075648 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 532790274560 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9778160 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚22665624 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 trainer/global_step‚699 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5888670 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚39547896704512 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9778160 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚37948082869423 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚37438203435008 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39081004005376 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚468 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3422517ÓÔFƒµ'ï ¯O HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 % _timestamp‚1.7883938101115956e+09 _runtime‚ 583.091728444 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚39081004005376 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 512602969088 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 512766854144 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12887464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12867940 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39081184929792 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚38568401036288 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚37948259981091 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9778158 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚38568418075648 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚38568401036288 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚39015106429952 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚38568418075648 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 532790274560 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9778160 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚22665624 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 trainer/global_step‚699 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5888670 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚39547896704512 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9778160 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚37948082869423 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚37438203435008 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39081004005376 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚468 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3422517 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚22646098 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12887464 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9310719 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9311187 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 509896473775 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 4341104646Y+(é0ð + _wandbruntime‚583  _runtime‚583—«¢2ñ µM VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚38000884664407 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9791656 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230104064 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚39602415046656 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9791654 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚38621807997952 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 510763505751 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚39602645150720 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22677586 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39069108258816 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12905482 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12905482 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39135285612032 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 % _timestamp‚1.7883938108013809e+09 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚38621807997952 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚37490104119296 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22697138 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 _step‚55 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 510603433235 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5896726 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 513313715712 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22677586 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 513313715712 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚39135285612032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 PŒR’¸‰ BDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12885932 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚37490121158656 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚12885932 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9791654 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9323851 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3426664 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚461 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚458 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9323390 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39069083355136 B%Í¿&ò ¹M VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚38000884664407 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9791656 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230104064 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚39602415046656 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9791654 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚38621807997952 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 510763505751 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚39602645150720 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22677586 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39069108258816 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12905482 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12905482 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39135285612032 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 % _timestamp‚1.7883938108013809e+09 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚38621807997952 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚37490104119296 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22697138 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 _step‚55 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 510603433235 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5896726 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 513313715712 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22677586 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 513313715712 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚39135285612032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12885932 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚37490121158656 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚12885932 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9791654 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9323851 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3426664 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚461 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚458 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9323390 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39069083355136 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚41786065280000 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13790416 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3657436 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚41786246204416 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 548332013568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9948591 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚41237914190848 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚42285208147456 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚41786246204416 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚42284978030080 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚41715197873152 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13770892 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _runtime‚ 621.380753236 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10452960 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚41237914190848 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 569805060608 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚41237897151488 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230117376 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3656974 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚41786065280000 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚41237897151488 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚41715172969472 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13770892 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24223850 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6291617 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7883938484007158e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10452958„ÇPw ³O XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚41237914190848 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 569805060608 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚41237897151488 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230117376 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3656974 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 F}N'B!@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚41786065280000 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚41237897151488 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚41715172969472 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13770892 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24223850 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6291617 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7883938484007158e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10452958 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚13790416 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 570010274304 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 545271618783 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 trainer/global_step‚749 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9949056 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚41786065280000 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13790416 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3657436 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚41786246204416 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 548332013568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9948591 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚41237914190848 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚42285208147456 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚41786246204416 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚42284978030080 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚41715197873152 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13770892 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _runtime‚ 621.380753236 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10452960 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801: ?(0‚ + _wandbruntime‚622  _runtime‚622Jd¨½&ƒ ·M MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚24274890 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚41769174798336 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚41291304113152 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10466454 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3661696 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6299679 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13808434 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _step‚59 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13788884 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 # _timestamp‚1.78839384908215e+09 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 trainer/global_step‚749 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9961372 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚41840165948928 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚40632145991815 _runtime‚ 622.062429604 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚41840346886656 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚41291287073792 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 549042773504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 570555215872 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 228006912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚40085990301696 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13808434 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚41291287073792 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 549042773504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚24255338 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10466454 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 548878875136 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚24255338 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚41840346886656 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 548878875136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9961831 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 203103232 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 545978578243 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10466456 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 546138650759 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24274890 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0„—ÎáÁ&„ »M VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚40631968879939 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚41291304113152 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚41769199702016 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚13788884 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3662152 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6299679 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13808434 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _step‚59 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13788884 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 # _timestamp‚1.78839384908215e+09 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 trainer/global_step‚749 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9961372 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚41840165948928 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚40632145991815 _runtime‚ 622.062429604 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚41840346886656 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚41291287073792 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 549042773504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 570555215872 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 228006912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚40085990301696 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13808434 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚41291287073792 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 549042773504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚24255338 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10466454 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 548878875136 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚24255338 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚41840346886656 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 548878875136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9961831 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 203103232 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 545978578243 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10466456 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 546138650759 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24274890 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚24274890 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚41769174798336 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚41291304113152 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10466454 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3661696 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 trainer/global_step‚799 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚45022555908608 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11127758 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10585670 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11127760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11127758 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚437 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42629958760448 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 580646763791 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 583897172992 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 _runtime‚ 659.77536849 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚44491126554624 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14693368 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚44491126554624 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚43210782635907 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14673844 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚25801602 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14673844 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43907393266688 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚43907410306048 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6694567 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚44415289316352 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚43907410306048 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚25821128 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3891537 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42629975799808 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚43907393266688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 583733287936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 580806836099 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚43210605524239 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚44491307479040 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚10586107 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14693368 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚434 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 607055087104 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚44415264412672Õ«ð~º'‘ ´O QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚43907393266688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 583733287936 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 580806836099 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚43210605524239 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚44491307479040 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚10586107 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14693368 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚434 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 607055087104 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚44415264412672 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 583733287936 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6694570 % _timestamp‚1.7883938867953181e+09 _step‚62 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11127760 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3891103 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 607266592256 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 trainer/global_step‚799 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚45022555908608 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11127758 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10585670 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11127760 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11127758 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚437 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42629958760448 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 580646763791 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 583897172992 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 _runtime‚ 659.77536849 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚44491126554624 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14693368 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚44491126554624 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚43210782635907 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14673844 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚25801602 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14673844 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43907393266688 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚43907410306048 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6694567 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚44415289316352 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚43907410306048 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚25821128 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3891537 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42629975799808 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032>R r¿’’ ‚(:‡ ŸóâÔ­ž  gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚175.818' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.1636 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ëšOü,“ ‚(:¡ ŸóâÔ¨‰  disk.rbd0.out‚0 disk.sda1.in‚0 disk.sda3.in‚0$ disk./.usageGB‚232.9597511291504 disk.rbd1.out‚3.171875 disk.nvme0c0n1.in‚0 disk.sda.out‚ 664.296875 network.recv‚283427 disk.drbd2080.out‚0 disk.rbd0.in‚0# disk.nvme0n1.out‚263.9365234375 disk.sda1.out‚0 disk.dm-0.out‚263.9365234375 disk.drbd2266.in‚0( proc.memory.availableMB‚ 2762.9453125* disk./.usagePercent‚15.634124364527857 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda.in‚0 disk.dm-1.out‚ 664.296875 proc.cpu.threads‚17 network.sent‚ 1.204292e+06 disk.nvme0n1.in‚0 disk.drbd2266.out‚0 disk.dm-1.in‚0 disk.sda3.out‚ 664.296875$ memory_percent‚83.13632011413574 disk.drbd2265.out‚0# proc.memory.rssMB‚ 2916.90234375 cpu‚22.55411513102673 disk.rbd1.in‚0 disk.dm-0.in‚0 disk.sda2.in‚0 disk.drbd2265.in‚0* proc.memory.percent‚17.803359031677246% disk.nvme0c0n1.out‚263.9365234375 disk.sda2.out‚0 disk.drbd1716.in‚0×F0” + _wandbruntime‚660  _runtime‚660æé_Ç"• ºM PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14691836 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚45077067697152 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43960783188992 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚44545408161280 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6702629 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚433 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10598360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 607801455616 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚25833090 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 581353723251 $ _timestamp‚1.788393887478443e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11141256 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 581513795767 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 584607932928 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 608010850304 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚43960800228352 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25852642 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11141256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚44545227223552 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209394688 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚43263407319223 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3895734 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚43960800228352 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚63 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 584444034560 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 584444034560 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234298368 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11141254 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14711386 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚436 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚45077301995520 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 584607932928 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3896167 ^KDeviceStatsMonitor.on_train_batBJùÜùch_end/inactive_split_bytes.large_pool.freed‚44469266241536 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14691836 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _runtime‚ 660.458741614 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚44545408161280 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10598796 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42681893523456M›ú“Ä&– ¾M TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚25833090 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 581353723251 $ _timestamp‚1.788393887478443e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11141256 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 581513795767 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 584607932928 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 608010850304 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚43960800228352 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25852642 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11141256 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚44545227223552 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209394688 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚43263407319223 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3895734 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚43960800228352 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚63 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 584444034560 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 584444034560 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234298368 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11141254 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14711386 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚436 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚45077301995520 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 584607932928 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3896167 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚44469266241536 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14691836 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _runtime‚ 660.458741614 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚44545408161280 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10598796 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42681893523456 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14691836 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚45077067697152 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43960783188992 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚44545408161280 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6702629 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚433 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10598360 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 607801455616 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚25833090 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 OoãÆ0ž + _wandbruntime‚698  _runtime‚698tõó†ÆŸ À ' lr-AdamW/pg2‚0.00011976077788983316 trainer/global_step‚849 % _timestamp‚1.7883939256560397e+09 _runtime‚ 698.636145899 _step‚65 ' lr-AdamW/pg1‚0.00011976077788983316 OÊ  Ä ' lr-AdamW/pg1‚0.00011976077788983316 ' lr-AdamW/pg2‚0.00011976077788983316 trainer/global_step‚849 % _timestamp‚1.7883939256560397e+09 _runtime‚ 698.636145899 _step‚65AÑõ¹0¡ + _wandbruntime‚698  _runtime‚698—fP¶'¢ °O YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 619298447360 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚47115355855872 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚45842043963315 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11802558 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11224479 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4126500 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 trainer/global_step‚849 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4126959 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 619462332416 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚47115380759552 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11802558 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240603136 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11224017 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11802560 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47759891999232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7097520 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 616021908799 _step‚66 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚15596320 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47196368753664 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215699456 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚45225861982208 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚46576889381888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚47759651396096 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15576796 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚45225844942848 _runtime‚ 698.636919439 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7097517 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚27379354 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚27398880 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 644511239680 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 $ _timestamp‚1.788393925656935e+09 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚47196187829248 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 644295540224 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 616181981107 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47196187829248 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 619298447360æ?›RË£ ´O SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11224017 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11802560 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47759891999232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7097520 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 616021908799 _step‚66 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚15596320 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47196368753664 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215699456 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚45225861982208 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚46576889381888 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚47759651396096 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15576796 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚45225844942848 _runtime‚ 698.636919439 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7097517 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚27379354 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚27398880 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 644511239680 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 $ _timestamp‚1.788393925656935e+09 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚47196187829248 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 644295540224 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 616181981107 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47196187829248 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 619298447360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 619298447360 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚47115355855872 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚45842043963315 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11802558 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11224479 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4126500 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 trainer/global_step‚849 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4126959 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 619462332416 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚47115380759552 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11802558 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240603136 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464B¯e°*0¤ + _wandbruntime‚699  _runtime‚699ÃwqvÁ&¥ »M NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11816054 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11816056 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4131131 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11816054 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚457 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15614338 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47250288498176 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 620173092352 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 645040929792 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚45894668646631 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 616888940775 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11236707 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚46630279304192 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚47814630815744 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47250469435904 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 620009193984 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚27410842 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚45277762666496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11816056 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 616728868259 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7105576 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _runtime‚ 699.329271102 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚47250469435904 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15594788 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚27430394 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚460 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4131588 trainer/global_step‚849 _step‚67 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚27410842 % _timestamp‚1.7883939263490674e+09 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11237167 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚47814398614528 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7105579 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚46630296343552 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47169357684736 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368Á‚Ò»Å&¦ ¿M XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11236707 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚46630279304192 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚47814630815744 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47250469435904 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 620009193984 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚27410842 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚45277762666496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11816056 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 616728868259 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7105576 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _runtime‚ 699.329271102 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚47250469435904 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15594788 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚27430394 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚460 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4131588 trainer/global_step‚849 _step‚67 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚27410842 % _timestamp‚1.7883939263490674e+09 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11237167 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚47814398614528 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7105579 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚46630296343552 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47169357684736 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚11816054 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11816056 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4131131 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11816054 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚457 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15614338 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47250288498176 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 620173092352 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 645040929792 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚45894668646631 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 616888940775 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464Cx;‘0§ + _wandbruntime‚699  _runtime‚699 xO¨ É 0loss/FBt1/assignment_loss‚0.2702886575843086 /loss/FBt2/detection_loss‚1.3162358514523078 %loss/total_loss‚79.04365412389339 /loss/FRt2/assignment_loss‚3.818156001442357 .loss/FRt3/detection_loss‚2.415016155184051 -loss/FBt1/detection_loss‚5.50719351094702 trainer/global_step‚849 $ _timestamp‚1.788393926350743e+09 .loss/FBt3/detection_loss‚3.067309629917145 epoch‚1 /loss/FRt2/detection_loss‚2.9061955657444503 .loss/FRt1/detection_loss‚3.349904521127765 1loss/SRqqt3/detection_loss‚1.6710343736906845 1loss/SRqqt4/detection_loss‚2.4894502339964237 /loss/FRt4/assignment_loss‚4.823433073361715 2loss/SRqqt2/assignment_loss‚2.7963832729267626 0loss/FBt3/assignment_loss‚0.4795041215606034 0loss/FBt4/assignment_loss‚0.5623626227206842 /loss/FBt4/detection_loss‚14.495939848944545 /loss/FRt3/assignment_loss‚4.101935374883958 2loss/SRqqt3/assignment_loss‚2.5597014860312144 /loss/FRt4/detection_loss‚3.8540196470419565 _runtime‚ 699.330373161 1loss/SRqqt1/assignment_loss‚2.457685880936109 1loss/FBt2/assignment_loss‚0.24176526899712386 0loss/SRqqt1/detection_loss‚3.405977087907302 0loss/SRqqt2/detection_loss‚4.815125132928479 _step‚68 /loss/FRt1/assignment_loss‚4.877927026052154 1loss/SRqqt4/assignment_loss‚2.761109778514275ÂZž4S© Í 1loss/SRqqt1/assignment_loss‚2.457685880936109 1loss/FBt2/assignment_loss‚0.24176526899712386 0loss/SRqqt1/detection_loss‚3.405977087907302 0loss/SRqqt2/detection_loss‚4.815125132928479 _step‚68 /loss/FRt1/assignment_loss‚4.877927026052154 1loss/SRqqt4/assignment_loss‚2.761109778514275 0loss/FBt1/assignment_loss‚0.2702886575843086 /loss/FBt2/detection_loss‚1.3162358514523078 %loss/total_loss‚79.04365412389339 /loss/FRt2/assignment_loss‚3.818156001442357 .loss/FRt3/detection_loss‚2.415016155184051 -loss/FBt1/detection_loss‚5.50719351094702 trainer/global_step‚849 $ _timestamp‚1.788393926350743e+09 .loss/FBt3/detection_loss‚3.067309629917145 epoch‚1 /loss/FRt2/detection_loss‚2.9061955657444503 .loss/FRt1/detection_loss‚3.349904521127765 1loss/SRqqt3/detection_loss‚1.6710343736906845 1loss/SRqqt4/detection_loss‚2.4894502339964237 /loss/FRt4/assignment_loss‚4.823433073361715 2loss/SRqqt2/assignment_loss‚2.7963832729267626 0loss/FBt3/assignment_loss‚0.4795041215606034 0loss/FBt4/assignment_loss‚0.5623626227206842 /loss/FBt4/detection_loss‚14.495939848944545 /loss/FRt3/assignment_loss‚4.101935374883958 2loss/SRqqt3/assignment_loss‚2.5597014860312144 /loss/FRt4/detection_loss‚3.8540196470419565 _runtime‚ 699.330373161DûŸ·ª ‚(:… ÌóâÔÚî§  gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚174.26' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.852 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÅÁN9« ‚(:® ÌóâÔ”ªþ  disk.sda.in‚0# proc.memory.rssMB‚ 2916.91015625 cpu‚22.72312042983368 disk.drbd2266.in‚0 disk.rbd1.out‚ 3.7041015625 disk.sda2.out‚0 disk.rbd0.in‚0) proc.memory.availableMB‚ 2762.27734375 disk.drbd2266.out‚0 disk.sda3.in‚0 disk.nvme0c0n1.in‚0& disk.nvme0c0n1.out‚266.96923828125 disk.drbd1716.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0! disk.dm-0.out‚266.96923828125 disk.sda2.in‚0 disk.nvme0n1.in‚0 disk.sda.out‚ 712.46484375* proc.memory.percent‚17.803406715393066$ disk./.usageGB‚232.9605255126953 disk.sda1.in‚0 disk.rbd1.in‚0 disk.drbd1716.out‚0 network.sent‚ 1.360503e+06) disk./.usagePercent‚15.63417633405731 disk.sda3.out‚ 712.46484375$ memory_percent‚83.14039707183838 proc.cpu.threads‚17 network.recv‚298662 disk.sda1.out‚0 disk.dm-1.out‚ 712.46484375 disk.dm-0.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0$ disk.nvme0n1.out‚266.96923828125 disk.drbd2265.in‚0µ†`’¬ ‚(:‡ ÛóâÔ££  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚160.469' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.0938 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ŒÛÏÀ:­ ‚(:¯ ÛóâÔߎ¦ disk.sda1.in‚0& disk.nvme0c0n1.out‚267.16064453125 disk.dm-1.out‚ 733.95703125 disk.sda3.out‚ 733.95703125! disk.dm-0.out‚267.16064453125 disk.sda2.in‚0 disk.nvme0n1.in‚0# memory_percent‚83.1378698348999) disk./.usagePercent‚15.63417505401964 disk.sda3.in‚0 disk.drbd1716.in‚0 network.sent‚ 1.446898e+06 network.recv‚304774 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.drbd2266.out‚0) proc.memory.availableMB‚ 2762.69140625 cpu‚22.775399649724708 disk.rbd1.out‚ 3.7041015625 disk.sda.out‚ 733.95703125 proc.cpu.threads‚17 disk.sda1.out‚0 disk.dm-1.in‚0 disk.rbd1.in‚0 disk.drbd1716.out‚0 disk.drbd2266.in‚0* proc.memory.percent‚17.803406715393066 disk.sda2.out‚0 disk.rbd0.out‚0$ disk.nvme0n1.out‚267.16064453125 disk.drbd2265.in‚0 disk.drbd2265.out‚0# proc.memory.rssMB‚ 2916.91015625 disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.rbd0.in‚0% disk./.usageGB‚232.96050643920898y!è#‘® ‚(:† êóâÔðåð  gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚175.325' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.065 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Œê½2¯ ‚(:§ êóâÔ‚¦Ø  disk.dm-1.out‚ 745.0390625 disk.drbd2265.out‚0 disk.rbd1.out‚ 3.845703125 disk.nvme0n1.in‚0) disk./.usagePercent‚15.63417505401964 disk.drbd2265.in‚0# disk.nvme0n1.out‚267.6708984375 disk.sda2.in‚0% disk./.usageGB‚232.96050643920898 disk.drbd2266.out‚0 disk.sda.out‚ 745.0390625 cpu‚22.825635534173717 network.recv‚307796 disk.nvme0c0n1.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 proc.cpu.threads‚17 disk.sda1.in‚0 disk.dm-1.in‚0 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.sda3.in‚0% disk.nvme0c0n1.out‚267.6708984375 disk.rbd0.in‚0$ memory_percent‚83.13684463500977' proc.memory.availableMB‚ 2762.859375 disk.dm-0.out‚267.6708984375 disk.sda3.out‚ 745.0390625 network.sent‚ 1.452005e+06 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.rbd1.in‚0 disk.dm-0.in‚0 disk.sda.in‚0 disk.rbd0.out‚0# proc.memory.rssMB‚ 2916.91015625* proc.memory.percent‚17.803406715393066ã}'50° + _wandbruntime‚737  _runtime‚737}öΑƱ À trainer/global_step‚899 % _timestamp‚1.7883939645287857e+09 _runtime‚ 737.509125189 _step‚69 ' lr-AdamW/pg1‚0.00011949053217586523 ' lr-AdamW/pg2‚0.00011949053217586523 ~Äʲ Ä ' lr-AdamW/pg1‚0.00011949053217586523 ' lr-AdamW/pg2‚0.00011949053217586523 trainer/global_step‚899 % _timestamp‚1.7883939645287857e+09 _runtime‚ 737.509125189 _step‚69EL`¬Ž0³ + _wandbruntime‚737  _runtime‚737øxCô ±O M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11861740 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/o±èȆô&versize_segments.current‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12477360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7500470 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 681762454528 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12477358 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7500467 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚47821748164608 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 654863606784 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 651557126115 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚12477360 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚49246402536448 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 737.509492031 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 654863606784 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚50496998248448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚49815447299072 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4361715 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚442 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _step‚70 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚49901430028288 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 655027491840 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 655027491840 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12477358 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚16499272 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚48473128179055 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚49246385497088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚899 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚28957106 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚49246402536448 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28976632 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚445 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16499272 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚49246385497088 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚50497234657280 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚49901249103872 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11862185 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚16479748 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486µR;»'µ µO E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚28957106 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚49246402536448 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28976632 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚445 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16499272 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚49246385497088 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚50497234657280 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚49901249103872 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11862185 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚16479748 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11861740 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12477360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7500470 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 681762454528 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12477358 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7500467 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚47821748164608 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 654863606784 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 651557126115 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚12477360 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚49246402536448 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 737.509492031 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 654863606784 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚50496998248448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚49815447299072 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4361715 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚442 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _step‚70 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚49901430028288 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 655027491840 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 655027491840 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12477358 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚16499272 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚48473128179055 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚49246385497088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚899 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416F£Ë`J0¶ + _wandbruntime‚738  _runtime‚738;ÝŒôÁ&· »M VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚16517290 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚49955530710528 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29008146 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16497740 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29008146 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4366028 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚48525752862163 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 655574353408 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚50551743810048 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚28988594 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚50551978108416 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚49869449127936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚28988594 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚49299775419392 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11874554 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12490854 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 trainer/global_step‚899 % _timestamp‚1.7883939652118053e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚49299792458752 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚49299792458752 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4366490 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 682504076800 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 682294682112 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234298368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 652264085783 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚49955530710528 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚49869474031616 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7508529 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7508526 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚47873665888256 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16517290 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 655738251776 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 652104013267 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚47873648848896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 655738251776 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12490856 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 655574353408 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11875019 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12490856 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12490854 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚49299775419392 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚48525929974039 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚462 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209394688 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 738.192196083a18F= ¸ ¿M ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29008146 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4366028 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚48525752862163 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 655574353408 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚50551743810048 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚28988594 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚50551978108416 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚49869449127936 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚28988594 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚49299775419392 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11874554 ;5ú ]ˆDeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12490854 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 trainer/global_step‚899 % _timestamp‚1.7883939652118053e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚49299792458752 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚49299792458752 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4366490 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 682504076800 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 682294682112 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234298368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 652264085783 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚49955530710528 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚49869474031616 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7508529 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7508526 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚47873665888256 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16517290 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 655738251776 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 652104013267 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚47873648848896 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 655738251776 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12490856 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 655574353408 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11875019 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12490856 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12490854 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚49299775419392 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚48525929974039 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚462 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209394688 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 738.192196083 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚16517290 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚49955530710528 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29008146 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16497740 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚51915898651648 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚12499521 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚52606310378496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚51915898651648 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚51104566618131 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 686932271123 trainer/global_step‚949 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7903417 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13152158 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚444 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚52606491302912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13152160 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 _runtime‚ 775.905200721 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚17402224 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚52515538742272 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚17402224 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚447 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7903420 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 718969363968 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 718757858816 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚50417634347008 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 686772198815 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚52606491302912 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚51104389506463 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 _step‚74 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚30534858 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 690592651264 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17382700 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚52606310378496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17382700 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 690592651264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚13152158 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12499074 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944/É,h#Å ´O bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 718969363968 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 718757858816 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚50417634347008 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 686772198815 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚52606491302912 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19524 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚51104389506463 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 _step‚74 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚30534858 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 690592651264 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17382700 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚52606310378496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17382700 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 690592651264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚13152158 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12499074 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 $ _timestamp‚1.788394002923153e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚53234533009920 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚51915881612288 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚51915898651648 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚12499521 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚52606310378496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚51915898651648 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚51104566618131 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 686932271123 trainer/global_step‚949 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7903417 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13152158 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚444 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚52606491302912 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13152160 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 935265O+I¢¥8944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 _runtime‚ 775.905200721 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚17402224 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚52515538742272 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚17402224 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚447 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7903420JÜô0Æ + _wandbruntime‚776  _runtime‚776ó[Á&Ç »M K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12511785 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 687479158275 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚51969288573952 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚53289276755456 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚52660591985152 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4600309 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 % _timestamp‚1.7883940036079752e+09 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚51157191301447 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚53289044554240 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚30566346 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 691139512832 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7911476 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚52660591985152 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚51969271534592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚50469552070656 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 691139512832 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚51969288573952 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 719503983104 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13165656 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚52569565474816 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚51157014189571 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17420242 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚51969271534592 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚30566346 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7911479 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 trainer/global_step‚949 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17400692 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 687639230791 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17400692 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚30585898 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17420242 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 _runtime‚ 776.588317857 _step‚75 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚443 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 691303411200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12512231 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚446 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4600752 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚50469535031296 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 691303411200 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13165656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 æµÅ&È ¿M J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13165654 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚52660411047424 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚52569540571136 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 719711280640 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚30585898 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17420242 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 _runtime‚ 776.588317857 _step‚75 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚443 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 691303411200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12512231 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚446 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4600752 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚50469535031296 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 691303411200 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13165656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12511785 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 687479158275 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚51969288573952 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚53289276755456 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚52660591985152 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4600309 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 % _timestamp‚1.7883940036079752e+09 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚51157191301447 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚53289044554240 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚30566346 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 691139512832 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7911476 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚52660591985152 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚51969271534592 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚50469552070656 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 691139512832 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚51969288573952 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 719503983104 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13165656 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚52569565474816 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚51157014189571 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17420242 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚51969271534592 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚30566346 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7911479 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 trainer/global_step‚949 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17400692 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 687639230791 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17400692 FDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _step‚78 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13135212 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13826960 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18305176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 725993925632 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚54585394766848 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚54585377727488 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8306367 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 trainer/global_step‚999 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _runtime‚ 814.519135285 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚55311371653120 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 722147343823 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚55311552577536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚53013520529408 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚434 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _step‚78 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13135212 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13826960 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18305176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 725993925632 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚54585394766848 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚54585377727488 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8306367 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 trainer/global_step‚999 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _runtime‚ 814.519135285 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚55311371653120 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 722147343823 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚55311552577536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513N÷gÛ+0Ø + _wandbruntime‚815  _runtime‚815÷ïÀ€Á&Ù »M FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚54638767649792 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 726704672256 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 726868570624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13147504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8314426 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 726868570624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 trainer/global_step‚999 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 726704672256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 236395520 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13840456 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13840454 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18303644 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13147953 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚32144098 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4833524 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚55365472322048 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚54638784689152 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 % _timestamp‚1.7883940422273147e+09 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚55365653259776 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18303644 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13840456 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18323194 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4833078 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚55269656918016 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 756649547776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚55365653259776 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚56026281562112 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32163650 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚56026517957632 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 722854303283 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009¼Aü½›Ú ¿M 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚55365472322048 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚54638784689152 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 % _timestamp‚1.7883940422273147e+09 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚55365653259776 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18303644 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13840456 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18323194 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4833078 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚55269656918016 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 756649547776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚55365653259776 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚56026281562112 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32163650 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚56026517957632 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 722854303283 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚54638767649792 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 726704672256 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 726868570624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13147504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8314426 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 726868570624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 trainer/global_step‚999 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 726704672256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 236395520 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13840456 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13840454 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18303644 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13147953 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚32144098 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4833524 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0OXzP0Û + _wandbruntime‚815  _runtime‚815?DUÜ Ï /loss/FBt2/detection_loss‚1.2865390263182332 1loss/SRqqt1/assignment_loss‚2.051025869342887 % _timestamp‚1.7883940422293446e+09 /loss/FRt3/assignment_loss‚4.483137355910407 .loss/FBt4/detection_loss‚10.17372422155581 %loss/total_loss‚70.36210042342219 2loss/SRqqt3/assignment_loss‚2.5271066779063807 0loss/FBt3/assignment_loss‚0.3278381715370213 .loss/FRt1/detection_loss‚2.589664434347678 /loss/FRt4/detection_loss‚2.7246841490268707 _runtime‚ 815.208981165 1loss/FBt2/assignment_loss‚0.34198220145698055 epoch‚1 0loss/FBt4/assignment_loss‚0.5554553323408503 1loss/SRqqt1/detection_loss‚2.2737733517746324 0loss/SRqqt2/detection_loss‚4.260496950199013 /loss/FBt1/detection_loss‚5.0088250099919565 /loss/FRt1/assignment_loss‚4.280649389695684 0loss/FRt2/assignment_loss‚4.1118142248570235 2loss/SRqqt2/assignment_loss‚2.9339637591483747 0loss/SRqqt4/detection_loss‚2.354318666534546 trainer/global_step‚999 _step‚80 1loss/FBt1/assignment_loss‚0.39893360761654845 /loss/FRt2/detection_loss‚2.9315096436218284 1loss/SRqqt3/detection_loss‚1.8641148960191964 .loss/FBt3/detection_loss‚3.703907923344997 /loss/FRt4/assignment_loss‚4.419247741699219 2loss/SRqqt4/assignment_loss‚2.5199677661443367 /loss/FRt3/detection_loss‚2.2394200530317097o¦6âYÝ Ó /loss/FRt4/assignment_loss‚4.419247741699219 2loss/SRqqt4/assignment_loss‚2.5199677661443367 /loss/FRt3/detection_loss‚2.2394200530317097 /loss/FBt2/detection_loss‚1.2865390263182332 1loss/SRqqt1/assignment_loss‚2.051025869342887 % _timestamp‚1.7883940422293446e+09 /loss/FRt3/assignment_loss‚4.483137355910407 .loss/FBt4/detection_loss‚10.17372422155581 %loss/total_loss‚70.36210042342219 2loss/SRqqt3/assignment_loss‚2.5271066779063807 0loss/FBt3/assignment_loss‚0.3278381715370213 .loss/FRt1/detection_loss‚2.589664434347678 /loss/FRt4/detection_loss‚2.7246841490268707 _runtime‚ 815.208981165 1loss/FBt2/assignment_loss‚0.34198220145698055 epoch‚1 0loss/FBt4/assignment_loss‚0.5554553323408503 1loss/SRqqt1/detection_loss‚2.2737733517746324 0loss/SRqqt2/detection_loss‚4.260496950199013 /loss/FBt1/detection_loss‚5.0088250099919565 /loss/FRt1/assignment_loss‚4.280649389695684 0loss/FRt2/assignment_loss‚4.1118142248570235 2loss/SRqqt2/assignment_loss‚2.9339637591483747 0loss/SRqqt4/detection_loss‚2.354318666534546 trainer/global_step‚999 _step‚80 1loss/FBt1/assignment_loss‚0.39893360761654845 /loss/FRt2/detection_loss‚2.9315096436218284 1loss/SRqqt3/detection_loss‚1.8641148960191964 .loss/FBt3/detection_loss‚3.703907923344997P¡ûò4’Þ ‚(:‡ ÄôâÔ¡ïÎ  gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.134' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.8268 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0òk p.ß ‚(:£ ÄôâÔ‡éŸ disk.dm-0.out‚270.6005859375 disk.sda.out‚ 836.59375 disk.sda3.in‚0 disk.rbd1.out‚ 4.2841796875 disk.nvme0c0n1.in‚0% disk.nvme0c0n1.out‚270.6005859375 disk.sda2.out‚0 network.sent‚ 1.646953e+06* disk./.usagePercent‚15.634196046637443% disk./.usageGB‚232.96081924438477 disk.sda2.in‚0 disk.nvme0n1.in‚0 disk.sda.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.dm-1.in‚0* proc.memory.percent‚17.803406715393066 network.recv‚335453 disk.rbd0.in‚0 disk.dm-1.out‚ 836.59375 disk.rbd1.in‚0) proc.memory.availableMB‚ 2523.26953125# proc.memory.rssMB‚ 2916.91015625 cpu‚23.09691374185703 disk.drbd1716.out‚0 disk.sda1.in‚0 disk.sda1.out‚0# memory_percent‚84.5991849899292 disk.dm-0.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.drbd1716.in‚0# disk.nvme0n1.out‚270.6005859375 disk.sda3.out‚ 836.59375 proc.cpu.threads‚17õß’’à ‚(:‡ ÓôâÔ”ÖÅ  gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚172.377' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4754 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0©÷ †8á ‚(:­ ÓôâÔ½ÏÍ  disk.sda1.out‚0 disk.dm-1.out‚ 855.90234375 disk.rbd0.in‚0 disk.drbd2265.out‚0# proc.memory.rssMB‚ 2916.91015625 proc.cpu.threads‚17 disk.sda1.in‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.nvme0n1.in‚0$ memory_percent‚84.59773063659668* proc.memory.percent‚17.803406715393066 network.recv‚341588 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0 disk.sda3.in‚0 disk.rbd1.in‚0 disk.nvme0c0n1.in‚0 disk.dm-0.in‚0 disk.dm-0.out‚270.7646484375 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda.in‚0* disk./.usagePercent‚15.634196046637443% disk.nvme0c0n1.out‚270.7646484375 disk.drbd1716.out‚0 disk.sda.out‚ 855.90234375 network.sent‚ 1.733561e+06 disk.sda3.out‚ 855.90234375# disk.nvme0n1.out‚270.7646484375 disk.rbd1.out‚ 4.2841796875 disk.drbd2080.in‚0 disk.drbd2266.in‚0( proc.memory.availableMB‚ 2523.5078125 cpu‚23.136832491701878% disk./.usageGB‚232.96081924438477 disk.drbd2080.out‚0Ó]S90â + _wandbruntime‚853  _runtime‚853ñXi>Çã Á ' lr-AdamW/pg2‚0.00011838379043131131 trainer/global_step‚1049 % _timestamp‚1.7883940802622876e+09 _runtime‚ 853.242210355 _step‚81 ' lr-AdamW/pg1‚0.00011838379043131131; …Ëä Å trainer/global_step‚1049 % _timestamp‚1.7883940802622876e+09 _runtime‚ 853.242210355 _step‚81 ' lr-AdamW/pg1‚0.00011838379043131131 ' lr-AdamW/pg2‚0.00011838379043131131Q©kMü0å + _wandbruntime‚853  _runtime‚853;@0à¸'æ ²O RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56366912161279 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19208128 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57915721628672 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796608 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚57254873842688 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚57254890882048 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 761559085056 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚56367089272947 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55609406711808 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _step‚82 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19188604 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 761722970112 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14501758 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚33709888 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚58708793712128 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚58016432927744 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚58016432927744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19188604 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚57254890882048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚55609389672448 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5063827 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14501760 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14501758 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 757522488831 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56366912161279 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19208128 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57915721628672 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796608 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚57254873842688 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚57254890882048 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 761559085056 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚56367089272947 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55609406711808 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 _step‚82 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19188604 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 761722970112 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14501758 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚33709888 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚58708793712128 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚58016432927744 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚58016432927744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19188604 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚57254890882048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚55609389672448 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5063827 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14501760 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14501758 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 757522488831 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 I8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚57308263764992 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚55661324435456 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 794029489152 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚467 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚58070533596672 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 trainer/global_step‚1049 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚83 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 758389520807 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 758229448291 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_end/reser&q³}Zved_bytes.small_pool.current‚ 434110464 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚58070714534400 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19226146 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19206596 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 762269831680 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚464 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚58763777850368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚57308263764992 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚33721850 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚57308280804352 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14515254 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚19226146 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚58763545649152 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57969723457536 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 793822191616 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19206596 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚33741402 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13785891 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56419713956263 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 762433730048 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8717376 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8717379 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚57308280804352 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚33721850 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623±Â3CÆ&ê ÀM UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5068979 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚57308263764992 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚55661324435456 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232201216 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 794029489152 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚467 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚58070533596672 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 trainer/global_step‚1049 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚83 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 758389520807 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 758229448291 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚58070714534400 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19226146 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19206596 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 762269831680 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚464 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚58763777850368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚57308263764992 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚33721850 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚57308280804352 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14515254 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚19226146 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚58763545649152 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57969723457536 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 793822191616 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19206596 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚33741402 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13785891 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56419713956263 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 762433730048 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8717376 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8717379 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297536 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚57308280804352 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚33721850 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623S­0ë + _wandbruntime‚853  _runtime‚853ÚøXrNì È 0loss/SRqqt3/detection_loss‚2.105650474538477 %loss/total_loss‚69.52975077357462 epoch‚1 trainer/global_step‚1049 /loss/FRt4/assignment_loss‚4.460249036550522 /loss/FBt4/assignment_loss‚0.756877560546193 /loss/FRt4/detection_loss‚3.0155149179956187 .loss/FBt1/detection_loss‚6.487224634849664 1loss/SRqqt1/assignment_loss‚2.122195177764263 /loss/FBt2/detection_loss‚1.2672426796826128 /loss/FRt1/assignment_loss‚4.101979325426385 -loss/FBt4/detection_loss‚8.32337528060783 2loss/SRqqt2/assignment_loss‚2.9289026610991535 1loss/SRqqt1/detection_loss‚2.4005716838083178 .loss/FRt1/detection_loss‚2.921777663078714 /loss/FRt2/assignment_loss‚4.150206981932075 0loss/SRqqt3/assignment_loss‚2.55347193273709 2loss/SRqqt4/assignment_loss‚2.3970867304929664 1loss/FBt2/assignment_loss‚0.30438079998023404 0loss/SRqqt2/detection_loss‚3.129847958090006 0loss/SRqqt4/detection_loss‚2.351438077698861 /loss/FBt3/detection_loss‚3.2447040872806237 /loss/FRt3/assignment_loss‚4.500636825638432 0loss/FBt3/assignment_loss‚0.5201321666942837 % _timestamp‚1.7883940809472337e+09 _runtime‚ 853.926834843 /loss/FRt2/detection_loss‚2.5196253116269713 0loss/FBt1/assignment_loss‚0.4175869311059642 .loss/FRt3/detection_loss‚2.549071874349348 _step‚84·›6èRí Ì /loss/FRt1/assignment_loss‚4.101979325426385 -loss/FBt4/detection_loss‚8.32337528060783 2loss/SRqqt2/assignment_loss‚2.9289026610991535 1loss/SRqqt1/detection_loss‚2.4005716838083178 .loss/FRt1/detection_loss‚2.921777663078714 /loss/FRt2/assignment_loss‚4.150206981932075 0loss/SRqqt3/assignment_loss‚2.55347193273709 2loss/SRqqt4/assignment_loss‚2.3970867304929664 1loss/FBt2/assignment_loss‚0.30438079998023404 0loss/SRqqt2/detection_loss‚3.129847958090006 0loss/SRqqt4/detection_loss‚2.351438077698861 /loss/FBt3/detection_loss‚3.2447040872806237 /loss/FRt3/assignment_loss‚4.500636825638432 0loss/FBt3/assignment_loss‚0.5201321666942837 % _timestamp‚1.7883940809472337e+09 _runtime‚ 853.926834843 /loss/FRt2/detection_loss‚2.5196253116269713 0loss/FBt1/assignment_loss‚0.4175869311059642 .loss/FRt3/detection_loss‚2.549071874349348 _step‚84 0loss/SRqqt3/detection_loss‚2.105650474538477 %loss/total_loss‚69.52975077357462 epoch‚1 trainer/global_step‚1049 /loss/FRt4/assignment_loss‚4.460249036550522 /loss/FBt4/assignment_loss‚0.756877560546193 /loss/FRt4/detection_loss‚3.0155149179956187 .loss/FBt1/detection_loss‚6.487224634849664 1loss/SRqqt1/assignment_loss‚2.122195177764263 /loss/FBt2/detection_loss‚1.2672426796826128TÂNê’î ‚(:‡ âôâÔŸé÷  gpu.0.gpu‚27.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚176.114' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.2228 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ëæ%3ï ‚(:¨ âôâÔù­Ç  disk.rbd1.out‚ 4.4560546875 disk.rbd0.out‚0 disk.sda1.in‚0 proc.cpu.threads‚17 disk.drbd2265.in‚0 disk.sda2.in‚0 network.sent‚ 1.7389e+06 network.recv‚345440 disk.rbd1.in‚0 disk.nvme0c0n1.in‚0 disk.drbd1716.out‚0 disk.rbd0.in‚0# disk.nvme0n1.out‚271.6123046875 disk.sda2.out‚0 disk.dm-0.in‚0 disk.dm-0.out‚271.6123046875 disk.sda.out‚ 871.45703125 disk.drbd2266.out‚0* disk./.usagePercent‚15.634210895074432% disk./.usageGB‚232.96104049682617 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚0" proc.memory.rssMB‚ 2916.9140625 disk.sda3.out‚ 871.45703125 disk.dm-1.out‚ 871.45703125( proc.memory.availableMB‚ 2523.4609375* proc.memory.percent‚17.803430557250977 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.sda1.out‚0# memory_percent‚84.5980167388916% disk.nvme0c0n1.out‚271.6123046875 disk.sda.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 cpu‚23.17622104844285,ùÒð ‚(:’ ñôâÔßÜí  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚141.047' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.209400000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æ™4ñ ‚(:© ñôâÔ®ÂÏ  disk.drbd2265.out‚0 disk.nvme0c0n1.in‚0$ memory_percent‚84.59980487823486 disk.rbd0.in‚0# disk.nvme0n1.out‚271.8271484375) proc.memory.availableMB‚ 2523.16796875 network.recv‚350782 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.sda1.in‚0* proc.memory.percent‚17.803430557250977 disk.sda2.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚0 disk.drbd2266.out‚0 proc.cpu.threads‚17* disk./.usagePercent‚15.634210895074432% disk./.usageGB‚232.96104049682617 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 882.8203125 disk.sda3.in‚0 disk.sda3.out‚ 882.8203125 disk.sda.out‚ 882.8203125 disk.rbd1.in‚0 disk.rbd1.out‚ 4.5419921875 disk.dm-0.in‚0 disk.dm-0.out‚271.8271484375" proc.memory.rssMB‚ 2916.9140625 cpu‚23.213929317647022 network.sent‚ 1.82462e+06 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.drbd2265.in‚0% disk.nvme0c0n1.out‚271.8271484375ÿ튒ò ‚(:‡ €õâÔˆ€‘  gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚179.682' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.9364 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0(°z+ó ‚(:  €õâÔÞžš disk.sda1.out‚0% disk./.usageGB‚232.96104049682617 disk.sda2.in‚0 disk.dm-0.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 895.58203125) proc.memory.availableMB‚ 2523.16796875* disk./.usagePercent‚15.634210895074432 disk.drbd2080.out‚0 disk.sda.in‚0* proc.memory.percent‚17.803430557250977 network.recv‚355931 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.rbd1.out‚ 4.6044921875 cpu‚23.250992873342796 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.sda3.out‚ 895.58203125 disk.rbd1.in‚0$ memory_percent‚84.59980487823486" proc.memory.rssMB‚ 2916.9140625 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.sda.out‚ 895.58203125 proc.cpu.threads‚17 disk.nvme0c0n1.in‚0! disk.nvme0c0n1.out‚ 272.296875 disk.dm-0.out‚ 272.296875 disk.drbd1716.in‚0 disk.sda1.in‚0 disk.sda3.in‚0 network.sent‚ 1.83116e+06 disk.nvme0n1.in‚0 disk.nvme0n1.out‚ 272.296875 disk.drbd2265.in‚0ãÜ(Q0ô + _wandbruntime‚891  _runtime‚891¹"ß"Æõ À ' lr-AdamW/pg1‚0.00011791723506982915 ' lr-AdamW/pg2‚0.00011791723506982915 trainer/global_step‚1099 $ _timestamp‚1.788394118673854e+09 _runtime‚ 891.653647781 _step‚85 æ­ìÊö Ä ' lr-AdamW/pg1‚0.00011791723506982915 ' lr-AdamW/pg2‚0.00011791723506982915 trainer/global_step‚1099 $ _timestamp‚1.788394118673854e+09 _runtime‚ 891.653647781 _step‚85ULÁ£ê0÷ + _wandbruntime‚891  _runtime‚891±=Åp]ø ²O LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚58998173488687 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60721494202368 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14410058 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 _runtime‚ 891.654979133 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 793057706147 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚86 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚59924369957888 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚60615837975552 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚58205275854848 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚35268114 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚59924386997248 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚59924386997248 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59924369957888 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 830508758528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 trainer/global_step‚1099 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 797124244480 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9112270 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚466 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚60615813071872 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚58998350600355 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5298257 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 792897633839 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23º¿â[ 763 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚58998173488687 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60721494202368 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14410058 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 _runtime‚ 891.654979133 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 793057706147 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚86 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408832 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚59924369957888 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚60615837975552 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚58205275854848 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚35268114 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚59924386997248 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚59924386997248 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59924369957888 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 830508758528 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211505152 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 trainer/global_step‚1099 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 797124244480 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9112270 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚466 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚60615813071872 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚58998350600355 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5298257 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 792897633839 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 I6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚58257193578496 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚59977759880192 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 % _timestamp‚1.7883941193625185e+09 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 797834991104 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 831248218112 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 797834991104 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚60775775809024 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚35299602 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60669839804416 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚35319154 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61500849529856 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 793764665815 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 trainer/global_step‚1099 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 _runtime‚ 892.342627367 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15190054 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60775775809024 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚35319154 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20109548 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5302998 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 _step‚87 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 797998889472 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚58257210617856 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14423327 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9120326 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15190056 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15190056 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚35299602 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5302528 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚59050975283671 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60775594871296 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14422854 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚59977776919552 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20129098 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623`|IÆ&ü ÀM VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚59977759880192 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 % _timestamp‚1.7883941193625185e+09 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 797834991104 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 831248218112 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 797834991104 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚60775775809024 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚35299602 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60669839804416 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚35319154 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61500849529856 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 793764665815 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 trainer/global_step‚1099 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 _runtime‚ 892.342627367 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15190054 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60775775809024 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚35319154 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20109548 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5302998 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 _step‚87 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 797998889472 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚58257210617856 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14423327 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9120326 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15190056 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15190056 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚35299602 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5302528 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚59050975283671 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60775594871296 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14422854 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚59977776919552 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20129098 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20129098 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238492672 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚58257193578496 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728W¢÷Ô…ý + —5n`)_wandbruntime‚892  _runtime‚892Š½‡ÎSþ Í %loss/total_loss‚66.07600603355674 2loss/SRqqt4/assignment_loss‚2.4963625407948786 1loss/SRqqt1/detection_loss‚1.8200419756178638 0loss/SRqqt4/detection_loss‚2.729947795977398 /loss/FBt2/detection_loss‚1.1154107271129015 0loss/FBt2/assignment_loss‚0.3217231019278157 /loss/FRt1/detection_loss‚2.5655287846636314 /loss/FRt3/detection_loss‚2.5322478848790366 .loss/FRt4/detection_loss‚2.837762155431382 _step‚88 0loss/FBt4/assignment_loss‚0.3588898400874034 /loss/FRt1/assignment_loss‚4.344691599790867 /loss/FRt4/assignment_loss‚4.594429093472501 1loss/SRqqt2/assignment_loss‚2.678862011179011 trainer/global_step‚1099 /loss/FRt2/assignment_loss‚4.225533544243156 2loss/SRqqt1/assignment_loss‚1.9651756376483058 1loss/FBt1/assignment_loss‚0.42844331544969466 0loss/SRqqt2/detection_loss‚4.113659948744672 1loss/SRqqt3/detection_loss‚1.8091503986242143 .loss/FBt1/detection_loss‚4.046166627407074 0loss/FBt3/assignment_loss‚0.4350803915403958 /loss/FRt2/detection_loss‚3.0752326677758974 epoch‚1 .loss/FBt3/detection_loss‚4.274730226823261 .loss/FBt4/detection_loss‚6.805836899922444 % _timestamp‚1.7883941193641222e+09 _runtime‚ 892.343744134 /loss/FRt3/assignment_loss‚4.241388116564069 2loss/SRqqt3/assignment_loss‚2.2597107478788683]Q—cWÿ Ñ epoch‚1 .loss/FBt3/detection_loss‚4.274730226823261 .loss/FBt4/detection_loss‚6.805836899922444 % _timestamp‚1.7883941193641222e+09 _runtime‚ 892.343744134 /loss/FRt3/assignment_loss‚4.241388116564069 2loss/SRqqt3/assignment_loss‚2.2597107478788683 %loss/total_loss‚66.07600603355674 2loss/SRqqt4/assignment_loss‚2.4963625407948786 1loss/SRqqt1/detection_loss‚1.8200419756178638 0loss/SRqqt4/detection_loss‚2.729947795977398 /loss/FBt2/detection_loss‚1.1154107271129015 0loss/FBt2/assignment_loss‚0.3217231019278157 /loss/FRt1/detection_loss‚2.5655287846636314 /loss/FRt3/detection_loss‚2.5322478848790366 .loss/FRt4/detection_loss‚2.837762155431382 _step‚88 0loss/FBt4/assignment_loss‚0.3588898400874034 /loss/FRt1/assignment_loss‚4.344691599790867 /loss/FRt4/assignment_loss‚4.594429093472501 1loss/SRqqt2/assignment_loss‚2.678862011179011 trainer/global_step‚1099 /loss/FRt2/assignment_loss‚4.225533544243156 2loss/SRqqt1/assignment_loss‚1.9651756376483058 1loss/FBt1/assignment_loss‚0.42844331544969466 0loss/SRqqt2/detection_loss‚4.113659948744672 1loss/SRqqt3/detection_loss‚1.8091503986242143 .loss/FBt1/detection_loss‚4.046166627407074 0loss/FBt3/assignment_loss‚0.4350803915403958 /loss/FRt2/detection_loss‚3.0752326677758974XbOÚŒ’€ ‚(:‡ õâÔðý   gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.156' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0312 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÜH1¬) ‚(:ž õâÔª­ * disk./.usagePercent‚15.634210895074432 disk.dm-1.in‚0 disk.sda.out‚ 909.73046875 disk.sda1.out‚0 disk.dm-1.out‚ 909.73046875 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda1.in‚0 network.sent‚ 1.837273e+06 disk.nvme0c0n1.in‚0 disk.rbd0.in‚0$ memory_percent‚84.60042476654053 proc.cpu.threads‚17 network.recv‚360122 disk.nvme0c0n1.out‚ 272.53125 disk.rbd0.out‚0 disk.nvme0n1.out‚ 272.53125 disk.sda.in‚0 disk.sda3.out‚ 909.73046875 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.nvme0n1.in‚0 disk.drbd2265.in‚0) proc.memory.availableMB‚ 2523.06640625* proc.memory.percent‚17.803430557250977 disk.dm-0.out‚ 272.53125% disk./.usageGB‚232.96104049682617 disk.sda3.in‚0 disk.drbd2265.out‚0 cpu‚23.286565003312695 disk.drbd2080.out‚0 disk.drbd2266.in‚0" proc.memory.rssMB‚ 2916.9140625 disk.drbd2266.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 4.6044921875 disk.dm-0.in‚0ÉŽÞ™’‚ ‚(:‡ žõâÔŸµ  gpu.0.gpu‚36.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚177.504' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.5008 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0é퀭1ƒ ‚(:¦ žõâÔØÈÞ disk.sda2.out‚0 disk.drbd1716.in‚0" proc.memory.rssMB‚ 2916.9140625 disk.dm-1.in‚0 disk.dm-0.out‚ 273.025390625 disk.sda.out‚ 931.66015625 disk.sda3.out‚ 931.66015625 network.sent‚ 1.922954e+06 network.recv‚365681$ disk.nvme0c0n1.out‚ 273.025390625 disk.rbd0.in‚0 disk.drbd2266.out‚0 disk.nvme0n1.in‚0 disk.rbd1.out‚ 4.74609375 disk.sda1.out‚0 disk.dm-1.out‚ 931.66015625 disk.dm-0.in‚0 disk.drbd1716.out‚0' proc.memory.availableMB‚ 2523.546875 disk.nvme0c0n1.in‚0 disk.sda2.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.drbd2266.in‚0$ memory_percent‚84.59749221801758 proc.cpu.threads‚17% disk./.usageGB‚232.96104431152344 disk.sda1.in‚0 disk.sda3.in‚0" disk.nvme0n1.out‚ 273.025390625* disk./.usagePercent‚15.634211151081967 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.rbd0.out‚0* proc.memory.percent‚17.803430557250977 cpu‚23.321969840821794 disk.drbd2080.in‚0 disk.drbd2265.out‚09†®’„ ‚(:‡ ­õâÔÈž¬  gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚168.801' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.7602 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ¶#÷8… ‚(:­ ­õâÔ÷†Ó  proc.cpu.threads‚17 disk.sda3.in‚0 disk.dm-0.in‚0 disk.drbd1716.out‚0 network.sent‚ 1.928218e+06 network.recv‚369098 disk.drbd2080.out‚0$ disk.nvme0n1.out‚273.31494140625 disk.sda1.out‚0# memory_percent‚84.5989465713501 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.dm-1.out‚ 940.08203125 disk.sda.in‚0 disk.sda.out‚ 940.08203125 disk.drbd2266.out‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 940.08203125) proc.memory.availableMB‚ 2523.30859375" proc.memory.rssMB‚ 2916.9140625 cpu‚23.355313562936136! disk.dm-0.out‚273.31494140625 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.dm-1.in‚0% disk./.usageGB‚232.96105194091797 disk.rbd1.in‚0 disk.nvme0c0n1.in‚0 disk.sda2.in‚0 disk.rbd0.out‚0* disk./.usagePercent‚15.634211663097034 disk.rbd1.out‚ 4.74609375& disk.nvme0c0n1.out‚273.31494140625 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.nvme0n1.in‚0* proc.memory.percent‚17.803430557250977ìˆ0† + _wandbruntime‚930  _runtime‚930L÷JèLJ Á _runtime‚ 930.199301132 _step‚89 ' lr-AdamW/pg1‚0.00011740247436128658 ' lr-AdamW/pg2‚0.00011740247436128658 trainer/global_step‚1149 % _timestamp‚1.7883941572195094e+09IQÄ.ˈ Å ' lr-AdamW/pg2‚0.00011740247436128658 trainer/global_step‚1149 % _timestamp‚1.7883941572195094e+09 _runtime‚ 930.199301132 _step‚89 ' lr-AdamW/pg1‚0.00011740247436128658YÒ~Ø°0‰ + _wandbruntime‚930  _runtime‚930~r»·'Š ±O dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚63315929418752 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 867463555072 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 828272778847 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚62593866073088 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚63426555476992 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚60801162037248 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 832689403904 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 $ _timestamp‚1.788394157220494e+09 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36845866 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 828432851155 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚20994508 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36865392 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚1149 _step‚90 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚63426555476992 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚62593883112448 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚61629434816095 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15851358 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 832853288960 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚63426736401408 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21014032 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5532558 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209408000 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚60801179076608 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9515220 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9515217 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64183602381824 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234311680 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 867672963072 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15047775 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5533010 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 832689403904 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15851360 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚62593883112448 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 q] »'‹ µO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 $ _timestamp‚1.788394157220494e+09 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36845866 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 828432851155 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚20994508 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36865392 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚1149 _step‚90 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚63426555476992 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚62593883112448 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚61629434816095 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15851358 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 832853288960 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚63426736401408 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21014032 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5532558 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209408000 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚60801179076608 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9515220 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9515217 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64183602381824 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234311680 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 867672963072 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15047775 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5533010 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23763 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885056 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 832689403904 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15851360 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚62593883112448 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚63315929418752 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 867463555072 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111668 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 828272778847 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚62593866073088 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚63426555476992 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚60801162037248 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 832689403904Zb^4§0Œ + _wandbruntime‚930  _runtime‚930òù ]Ú ¼M XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚91 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 829139810823 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36896906 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21012500 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚62647255995392 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚61682059499203 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚63480837083648 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 833564048896 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15864854 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚63480656145920 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 225909760 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚62647273034752 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5537710 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 833400150528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5537267 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 833564048896 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9523279 QCDeviceStatsMonitor.on_train_bat¾Ò;©èch_end/inactive_split_bytes.all.peak‚ 315496448 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36877354 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21012500 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 828979738307 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚36896906 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883941579050353e+09 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚62647273034752 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21032050 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚63369906343936 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚61682236611079 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15060989 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21032050 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚446 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚64238120381440 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚64238346291200 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚62647255995392 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚60853096800256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚443 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚63369931247616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚60853079760896 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9523276 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36877354 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15864854 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚63480837083648 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15864856 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15060543 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15864856 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944­Œ‚Æ&Ž ÀM K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15060543 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15864856 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _step‚91 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 829139810823 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36896906 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21012500 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚62647255995392 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23763 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚61682059499203 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚63480837083648 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 833564048896 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15864854 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚63480656145920 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 225909760 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚62647273034752 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5537710 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 833400150528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5537267 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 833564048896 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9523279 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36877354 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23763 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21012500 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 828979738307 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚36896906 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7883941579050353e+09 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚62647273034752 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21032050 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚63369906343936 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚61682236611079 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15060989 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21032050 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚446 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚64238120381440 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚64238346291200 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚62647255995392 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚60853096800256 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚443 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚63369931247616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚60853079760896 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9523276 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36877354 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15864854 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚63480837083648 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15864856[ÍC¿0 + _wandbruntime‚930  _runtime‚930…¿W Ñ 1loss/SRqqt2/detection_loss‚3.6197960896819246 /loss/FBt2/detection_loss‚0.9804976175440154 .loss/FBt4/detection_loss‚7.416078226132826 2loss/SRqqt3/assignment_loss‚2.3123972473813477 epoch‚1 trainer/global_step‚1149 % _timestamp‚1.7883941579067326e+09 2loss/SRqqt4/assignment_loss‚2.2939970526099205 .loss/FBt1/detection_loss‚4.509162704504671 _runtime‚ 930.886351887 /loss/FRt4/assignment_loss‚4.451648441644815 0loss/FBt4/assignment_loss‚0.5850986063259452 /loss/FRt4/detection_loss‚3.2153893884940024 0loss/SRqqt3/detection_loss‚1.887699557588898 /loss/FBt3/detection_loss‚3.1160770777576943 .loss/FRt1/detection_loss‚2.993809715750512 /loss/FRt3/detection_loss‚2.2681156504154205 %loss/total_loss‚64.90850348664827 _step‚92 1loss/FBt1/assignment_loss‚0.42636177314922125 0loss/FRt2/assignment_loss‚3.8838872530881097 2loss/SRqqt1/assignment_loss‚1.9774127348705575 2loss/SRqqt2/assignment_loss‚2.7839669363171446 1loss/FBt3/assignment_loss‚0.29257835915375513 0loss/SRqqt1/detection_loss‚2.203238580720844 0loss/FRt1/assignment_loss‚3.9581751836819596 .loss/FRt3/assignment_loss‚4.24243661403656 1loss/FBt2/assignment_loss‚0.27714737683641033 /loss/FRt2/detection_loss‚2.7420787693823083 0loss/SRqqt4/detection_loss‚2.471452529579401&E œ[‘ Õ 2loss/SRqqt4/assignment_loss‚2.2939970526099205 .loss/FBt1/detection_loss‚4.509162704504671 _runtime‚ 930.886351887 /loss/FRt4/assignment_loss‚4.451648441644815 0loss/FBt4/assignment_loss‚0.5850986063259452 /loss/FRt4/detection_loss‚3.2153893884940024 0loss/SRqqt3/detection_loss‚1.887699557588898 /loss/FBt3/detection_loss‚3.1160770777576943 .loss/FRt1/detection_loss‚2.993809715750512 /loss/FRt3/detection_loss‚2.2681156504154205 %loss/total_loss‚64.90850348664827 _step‚92 1loss/FBt1/assignment_loss‚0.42636177314922125 0loss/FRt2/assignment_loss‚3.8838872530881097 2loss/SRqqt1/assignment_loss‚1.9774127348705575 2loss/SRqqt2/assignment_loss‚2.7839669363171446 1loss/FBt3/assignment_loss‚0.29257835915375513 0loss/SRqqt1/detection_loss‚2.203238580720844 0loss/FRt1/assignment_loss‚3.9581751836819596 .loss/FRt3/assignment_loss‚4.24243661403656 1loss/FBt2/assignment_loss‚0.27714737683641033 /loss/FRt2/detection_loss‚2.7420787693823083 0loss/SRqqt4/detection_loss‚2.471452529579401 1loss/SRqqt2/detection_loss‚3.6197960896819246 /loss/FBt2/detection_loss‚0.9804976175440154 .loss/FBt4/detection_loss‚7.416078226132826 2loss/SRqqt3/assignment_loss‚2.3123972473813477 epoch‚1 trainer/global_step‚1149 % _timestamp‚1.7883941579067326e+09\éΔ¬’ ‚(:’ ¼õâԩÿ  gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚133.733' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.746599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0äC2÷-“ ‚(:¢ ¼õâÔÏÙ÷ * proc.memory.percent‚17.803430557250977 network.recv‚374474 disk.sda.out‚ 948.0703125 disk.dm-1.out‚ 948.0703125 network.sent‚ 1.934716e+06$ disk./.usageGB‚232.9610595703125 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.dm-0.out‚ 274.033203125 disk.sda2.out‚0 disk.sda3.in‚0 proc.cpu.threads‚17 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.nvme0c0n1.in‚0" disk.nvme0n1.out‚ 274.033203125 disk.dm-1.in‚0 disk.rbd1.in‚0$ memory_percent‚84.60073471069336$ disk.nvme0c0n1.out‚ 274.033203125 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.sda3.out‚ 948.0703125 disk.rbd1.out‚ 4.80859375' proc.memory.availableMB‚ 2523.015625 disk.dm-0.in‚0 disk.sda2.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.drbd2265.out‚0 cpu‚23.388363472403952* disk./.usagePercent‚15.634212175112102" proc.memory.rssMB‚ 2916.9140625£ ‘” ‚(:† ËõâÔà’Ø  gpu.0.gpu‚45.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚139.685' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.937 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0q(.k1• ‚(:¦ ËõâÔùËœ disk.sda1.out‚0 disk.rbd1.in‚0 disk.nvme0c0n1.in‚0$ disk.nvme0c0n1.out‚ 274.451171875* disk./.usagePercent‚15.634212943134706 disk.drbd1716.in‚0 disk.rbd0.in‚0* proc.memory.percent‚17.803430557250977$ disk./.usageGB‚232.9610710144043" disk.nvme0n1.out‚ 274.451171875 disk.drbd2266.out‚0 disk.dm-1.out‚ 959.33203125 disk.sda3.out‚ 959.33203125 network.recv‚380576 disk.sda.out‚ 959.33203125 disk.drbd2265.out‚0 disk.rbd1.out‚ 4.80859375 proc.cpu.threads‚17 network.sent‚ 2.021174e+06 disk.nvme0n1.in‚0 disk.sda.in‚0 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.dm-0.out‚ 274.451171875 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0" proc.memory.rssMB‚ 2916.9140625 disk.sda1.in‚0) proc.memory.availableMB‚ 2523.01953125 cpu‚23.42005024771896 disk.dm-0.in‚0 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.sda3.in‚0$ memory_percent‚84.60071086883545úNø0– + _wandbruntime‚968  _runtime‚968K¨ðBÆ— À ' lr-AdamW/pg1‚0.00011683993809540851 ' lr-AdamW/pg2‚0.00011683993809540851 trainer/global_step‚1199 $ _timestamp‚1.788394195507576e+09 _runtime‚ 968.487397148 _step‚93ÊWٚʘ Ä $ _timestamp‚1.788394195507576e+09 _runtime‚ 968.487397148 _step‚93 ' lr-AdamW/pg1‚0.00011683993809540851 ' lr-AdamW/pg2‚0.00011683993809540851 trainer/global_step‚1199]*fzÀ0™ + _wandbruntime‚968  _runtime‚968&#‡ð·'š ±O MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚66131797676032 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚16526158 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚66131616751616 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚65263362188288 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16526160 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21916984 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚65263379227648 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16526158 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16526160 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21897460 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63397048219648 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚66015995958272 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15685780 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240603136 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5767162 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21897460 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚65263379227648 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _step‚94 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21916984 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚65263362188288 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚38443144 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9918167 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38423618 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9918170 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66920667161088 trainer/global_step‚1199 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5767610 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚66016020861952 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 868418448384 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚66131616751616 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 868254563328 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15685329 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚66131797676032 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 863807996163 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 863647923855 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66920907764224 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚64260873255171 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚64260696143503 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568à¼O…Ô› µO M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚38443144 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9918167 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38423618 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885056 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422323712 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9918170 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66920667161088 trainer/global_step‚1199 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23486 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5767610 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 d+¢¾ç#QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚66016020861952 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 868418448384 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19526 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚66131616751616 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 868254563328 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15685329 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072308 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚66131797676032 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820801 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 863807996163 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19526 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23486 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 863647923855 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23763 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66920907764224 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚64260873255171 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚64260696143503 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚66131797676032 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚16526158 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚66131616751616 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚65263362188288 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16526160 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21916984 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚65263379227648 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16526158 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504009 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924416 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16526160 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352658944 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21897460 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924416 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63397048219648 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422323712 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚66015995958272 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15685780 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240603136 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19524 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5767162 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21897460 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚65263379227648 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _step‚94 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21916984 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352658944 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚65263362188288^Tösw0œ + _wandbruntime‚969  _runtime‚969} ÙeÂ& ¼M J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352658944 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚65316752110592 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16539656 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 969.172518955 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21915452 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5772357 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38474658 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21915452 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9926229 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚66975649886208 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15698155 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38474658 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚66185717420544 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9926226 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚66185898358272 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚65316769149952 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚65316752110592 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚63448965943296 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21935002 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚431 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5771929 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63448982982656 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16539654 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚65316769149952 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 868965309952 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚66070022690816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 905424092160 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 864514955831 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 % _timestamp‚1.7883941961924157e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21935002 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚428 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚66069997787136 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚64313320826611 trainer/global_step‚1199 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15698586 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15698155 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38474658 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚66185717420544 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9926226 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚66185898358272 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23486 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504009 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚65316769149952 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚65316752110592 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚63448965943296 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21935002 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚431 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5771929 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63448982982656 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072516 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352658944 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16539654 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚65316769149952 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 868965309952 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚66070022690816 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 905424092160 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 864514955831 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 % _timestamp‚1.7883941961924157e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21935002 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180937728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚428 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚66069997787136 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚64313320826611 trainer/global_step‚1199 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15698586 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422323712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19550 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 969.172518955 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21915452 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180937728 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898368 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111876 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898368 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422323712 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5772357 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820801 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19550 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19552 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38474658 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21915452 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9926229 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚66975649886208 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 E&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_1_of_1‚0.38248321413993835 *jet/accuracy_1_of_4‚0.8018487095832825 particle/accuracy_2_of_4‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.27119573950767517 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.19166666269302368 *jet/accuracy_1_of_6‚0.8684210777282715 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.28539034724235535 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.013265572488307953 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2857142984867096 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 +jet/accuracy_2_of_2‚0.07283209264278412 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.38954222202301025 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.02469135820865631 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.03792425990104675 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9869444370269775 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3462814688682556 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.28464803099632263 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0028076465241611004 particle/accuracy_5_of_5‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.28737717866897583 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.08205516636371613 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.11221197992563248 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.28402775526046753 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.3333333432674408 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9811546206474304 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.06786877661943436 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.962202250957489 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.2892577350139618 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.18527796864509583 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.39621445536613464 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01297266036272049 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10884770005941391 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3209877014160156 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9880840182304382 +jet/accuracy_3_of_6‚0.07017543911933899 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2617189586162567 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.12204482406377792 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.378888875246048 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18053969740867615 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3596683442592621 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.02778473123908043 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.35691383481025696 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2953719198703766 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.3744995892047882 )loss/val_total_loss‚63.55082054086754 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 jet/accuracy_6_of_6‚0 particle/accuracy_6_of_7‚1 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.41927629709243774 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9945031404495239 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9946109652519226 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3101113438606262 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3734602630138397 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 particle/accuracy_1_of_3‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.02666666731238365 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.41661372780799866 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.007012676447629929 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.1646258533000946 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9879032373428345 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2633853852748871 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9985975027084351 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.06114756688475609 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.3235294222831726 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.19471442699432373 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.37658727169036865 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.30000001192092896 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.36649197340011597 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.08863920718431473 *jet/accuracy_1_of_2‚0.6165058612823486 epoch‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_4_of_5‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 6 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.997067391872406 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_1_of_7‚1 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.12315289676189423 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 8 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.02777777798473835 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.24274972081184387 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07338987290859222 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.15814131498336792 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.1666666716337204 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14276212453842163 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 5 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.96132892370224 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.1944444477558136 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.36267009377479553 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.16657142341136932 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.035947710275650024 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.12901149690151215 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9886419773101807 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9774824380874634 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9651239514350891 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2616167366504669 particle/accuracy_1_of_5‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17013555765151978 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.029697900637984276 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.11183498054742813 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2710239887237549 particle/accuracy_1_of_2‚1 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2421294003725052 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5175438523292542 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2982456088066101 _runtime‚1011.140602851 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04592593014240265 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.11874417215585709 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03035190887749195 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_4‚0.23950017988681793 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.020979106426239014 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 *jet/accuracy_2_of_6‚0.3684210479259491 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9239463806152344 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3168832063674927 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 $ _timestamp‚1.788394238158785e+09 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10277777910232544 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0023623330052942038 jet/accuracy_4_of_6‚0 :"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.30000001192092896 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.32692307233810425 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.10392143577337265 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.07755763828754425 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.08610048145055771 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_7‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.31555187702178955 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17467716336250305 6"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3363057076931 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.06405304372310638 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.09143519401550293 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.09452993422746658 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.26126474142074585 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9905838966369629 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.15595237910747528 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6305314898490906 6 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.995582103729248 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34698814153671265 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9986852407455444 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.3202380836009979 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9823848009109497 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.171875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2236662358045578 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8"Purity/*FRt0SRqqt3FBt/event_purity‚0.032738097012043 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.13333334028720856 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9934832453727722 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01224489789456129 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_5‚0.8455787897109985 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 trainer/global_step‚1207 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001635678461752832 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.31292662024497986 7"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.28980752825737 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1897907853126526 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2341269999742508 _step‚97 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.31416404247283936 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.925198495388031 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.12320346385240555 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1261364221572876 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04031001776456833 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_7_of_7‚0.5 ;"Purity/*FRt*SRqqt3FBt/event_purity‚0.031609196215867996 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.001102920388802886 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1170080229640007 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 jet/accuracy_7_of_7‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.095238097012043 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.32953158020973206 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.26734277606010437 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8298156261444092 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.05418489873409271 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.988098680973053 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.33028092980384827 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9971633553504944 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03192925825715065 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.985169529914856 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_2_of_5‚0.2846869230270386 particle/accuracy_4_of_4‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3925406336784363 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.012914782389998436 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.34040367603302 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3214285671710968 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.1875 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 jet/accuracy_6_of_7‚0 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.35851579904556274 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.21251578629016876 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9802631735801697 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +particle/sensitivity‚0.2820741534233093 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4168202877044678 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2864583432674408 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt4SRqqt*FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_4_of_6‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2083333283662796 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.33028092980384827 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2719298303127289 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3079528510570526 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08195769041776657 particle/accuracy_2_of_5‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 6 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.089384526014328 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.2717391550540924 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3330366909503937 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3541474938392639 particle/accuracy_1_of_6‚1 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2083333283662796 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.27261030673980713 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.095238097012043 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.08867169916629791 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995391368865967 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.30416667461395264 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9788734912872314 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.986194908618927 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9907899498939514 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3333333432674408 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_7‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.06405304372310638 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_5‚1 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5476190447807312 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.09982369840145111 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2575053870677948 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.27591249346733093 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3055555522441864 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3510393500328064 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +jet/accuracy_2_of_3‚0.16132670640945435 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2548055350780487 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4104161560535431 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.235939159989357 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1285804659128189 +"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.25 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.10610365867614746 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9176955223083496 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.12005872279405594 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2909140884876251 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1316872537136078 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879914522171 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 jet/accuracy_3_of_7‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.39650535583496094 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/R>a'!4FRt*SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.28605571389198303 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.11043164879083633 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.29629626870155334 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 'particle/f_score‚0.3410695493221283 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0051695615984499454 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.5476190447807312 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2317548394203186 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3657738268375397 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 +particle/specificity‚0.9312033653259277 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_5_of_7‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.10762770473957062 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.3125 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1295025646686554 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_4_of_5‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.14665549993515015 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9908039569854736 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2502845227718353 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.12885428965091705 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_2_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.38349995017051697 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0020375435706228018 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.21023796498775482 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5833333134651184 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 particle/accuracy_1_of_1‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.1216450184583664 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.14141415059566498 *jet/accuracy_1_of_3‚0.7436853051185608 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 8"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.294298380613327 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_5‚0.042789556086063385 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_7‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9842046499252319 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.05695212259888649 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 8 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.02777777798473835 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,jet/accuracy_3_of_4‚0.031046753749251366 particle/accuracy_2_of_2‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 particle/accuracy_6_of_6‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3003939688205719 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9919354915618896 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9587963223457336 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.032401952892541885 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9868409633636475 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.31758248805999756 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.015753870829939842 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12409082055091858 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.299977570772171 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.08053461462259293 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.05488215759396553 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.2805556058883667 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.12403059005737305 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.36742356419563293 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.11377368122339249 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6280180811882019 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8124042749404907 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2725338041782379 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9937472343444824 :"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.48245611786842346 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9953696131706238 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.23288032412528992 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3223813772201538 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902ˆä^ª ¨¥ :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14276212453842163 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 5 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.96132892370224 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.1944444477558136 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.36267009377479553 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.16657142341136932 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.035947710275650024 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.12901149690151215 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9886419773101807 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9774824380874634 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9651239514350891 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2616167366504669 particle/accuracy_1_of_5‚1 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17013555765151978 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.029697900637984276 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.11183498054742813 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2710239887237549 particle/accuracy_1_of_2‚1 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.2421294003725052 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5175438523292542 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2982456088066101 _runtime‚1011.140602851 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04592593014240265 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.11874417215585709 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03035190887749195 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_4‚0.23950017988681793 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.020979106426239014 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 *jet/accuracy_2_of_6‚0.3684210479259491 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9239463806152344 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3168832063674927 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 $ _timestamp‚1.788394238158785e+09 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10277777910232544 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0023623330052942038 jet/accuracy_4_of_6‚0 :"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.30000001192092896 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.32692307233810425 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.10392143577337265 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.07755763828754425 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.08610048145055771 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_7‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.31555187702178955 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17467716336250305 6"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3363057076931 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.06405304372310638 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.09143519401550293 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.09452993422746658 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.26126474142074585 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9905838966369629 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.15595237910747528 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6305314898490906 6 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.995582103729248 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34698814153671265 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9986852407455444 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.3202380836009979 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9823848009109497 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.171875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_6‚1 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2236662358045578 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 8"Purity/*FRt0SRqqt3FBt/event_purity‚0.032738097012043 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.13333334028720856 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9934832453727722 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01224489789456129 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_5‚0.8455787897109985 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 trainer/global_step‚1207 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001635678461752832 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.31292662024497986 7"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.28980752825737 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1897907853126526 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2341269999742508 _step‚97 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.31416404247283936 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.925198495388031 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.12320346385240555 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1261364221572876 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04031001776456833 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_7_of_7‚0.5 ;"Purity/*FRt*SRqqt3FBt/event_purity‚0.031609196215867996 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.001102920388802886 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1170080229640007 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 jet/accuracy_7_of_7‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.095238097012043 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.32953158020973206 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.26734277606010437 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8298156261444092 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt4SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.05418489873409271 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.988098680973053 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.33028092980384827 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9971633553504944 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03192925825715065 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.985169529914856 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_2_of_5‚0.2846869230270386 particle/accuracy_4_of_4‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3925406336784363 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.012914782389998436 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.34040367603302 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3214285671710968 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.1875 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 jet/accuracy_6_of_7‚0 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.35851579904556274 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.21251578629016876 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9802631735801697 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +particle/sensitivity‚0.2820741534233093 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4168202877044678 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2864583432674408 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt4SRqqt*FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_4_of_6‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2083333283662796 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.33028092980384827 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2719298303127289 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3079528510570526 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08195769041776657 particle/accuracy_2_of_5‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 6 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.089384526014328 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.2717391550540924 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3330366909503937 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3541474938392639 particle/accuracy_1_of_6‚1 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2083333283662796 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.27261030673980713 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.095238097012043 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.08867169916629791 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995391368865967 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.30416667461395264 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9788734912872314 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.986194908618927 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9907899498939514 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3333333432674408 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_7‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.06405304372310638 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_5‚1 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5476190447807312 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.09982369840145111 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2575053870677948 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.27591249346733093 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3055555522441864 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3510393500328064 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 +jet/accuracy_2_of_3‚0.16132670640945435 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2548055350780487 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4104161560535431 6 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.235939159989357 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3333333432674408 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1285804659128189 +"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.25 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.10610365867614746 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9176955223083496 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.12005872279405594 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2909140884876251 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1316872537136078 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879914522171 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 jet/accuracy_3_of_7‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.39650535583496094 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.28605571389198303 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.11043164879083633 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.29629626870155334 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 'particle/f_score‚0.3410695493221283 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0051695615984499454 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.5476190447807312 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2317548394203186 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3657738268375397 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 +particle/specificity‚0.9312033653259277 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_5_of_7‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.10762770473957062 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.3125 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1295025646686554 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_4_of_5‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.14665549993515015 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9908039569854736 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2502845227718353 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.12885428965091705 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_2_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.38349995017051697 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0020375435706228018 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.21023796498775482 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5833333134651184 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 particle/accuracy_1_of_1‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.1216450184583664 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.14141415059566498 *jet/accuracy_1_of_3‚0.7436853051185608 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 8"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.294298380613327 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_5‚0.042789556086063385 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_7‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9842046499252319 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.05695212259888649 ,&Purity/4FRt3SRqqt4FÌ”HTË3Bt/event_proportion‚0 8 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.02777777798473835 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,jet/accuracy_3_of_4‚0.031046753749251366 particle/accuracy_2_of_2‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 particle/accuracy_6_of_6‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3003939688205719 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9919354915618896 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9587963223457336 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.032401952892541885 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9868409633636475 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.31758248805999756 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.015753870829939842 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12409082055091858 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.299977570772171 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.08053461462259293 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.05488215759396553 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.2805556058883667 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.12403059005737305 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.36742356419563293 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.11377368122339249 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6280180811882019 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8124042749404907 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2725338041782379 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9937472343444824 :"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.48245611786842346 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9953696131706238 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.23288032412528992 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3223813772201538 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2249651402235031 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_1_of_1‚0.38248321413993835 *jet/accuracy_1_of_4‚0.8018487095832825 particle/accuracy_2_of_4‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.27119573950767517 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.19166666269302368 *jet/accuracy_1_of_6‚0.8684210777282715 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.28539034724235535 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.013265572488307953 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2857142984867096 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 +jet/accuracy_2_of_2‚0.07283209264278412 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.38954222202301025 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.02469135820865631 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.03792425990104675 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9869444370269775 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3462814688682556 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.28464803099632263 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0028076465241611004 particle/accuracy_5_of_5‚1 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.28737717866897583 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.08205516636371613 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.11221197992563248 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.28402775526046753 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.3333333432674408 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9811546206474304 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.06786877661943436 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.962202250957489 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.2892577350139618 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.18527796864509583 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.39621445536613464 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.01297266036272049 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10884770005941391 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.3209877014160156 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9880840182304382 +jet/accuracy_3_of_6‚0.07017543911933899 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2617189586162567 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.12204482406377792 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.378888875246048 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18053969740867615 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.3596683442592621 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.02778473123908043 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.35691383481025696 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2953719198703766 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.3744995892047882 )loss/val_total_loss‚63.55082054086754 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 jet/accuracy_6_of_6‚0 particle/accuracy_6_of_7‚1 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.41927629709243774 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9945031404495239 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9946109652519226 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3101113438606262 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3734602630138397 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 particle/accuracy_1_of_3‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.02666666731238365 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.41661372780799866 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.007012676447629929 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.1646258533000946 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9879032373428345 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2633853852748871 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9985975027084351 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.06114756688475609 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.3235294222831726 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.19471442699432373 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.37658727169036865 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.30000001192092896 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.36649197340011597 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.08863920718431473 *jet/accuracy_1_of_2‚0.6165058612823486 epoch‚1 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_4_of_5‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 6 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.997067391872406 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_1_of_7‚1 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.12315289676189423 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 8 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.02777777798473835 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.24274972081184387 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07338987290859222 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.15814131498336792 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.1666666716337204 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0aAE/g’« ‚(:‡ ‡öâÔ®š­  gpu.0.gpu‚20.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚34.0 gpu.0.powerWatts‚123.123' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.6246 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0c ¢N)¬ ‚(:ž ‡öâÔÆ“Á  disk.sda.in‚0 disk.sda.out‚ 1007.0625 disk.nvme0c0n1.in‚0 disk.drbd2080.out‚0$ disk./.usageGB‚233.0807342529297 disk.drbd2265.in‚0* proc.memory.percent‚17.209625244140625 proc.cpu.threads‚17 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.dm-1.in‚0 disk.sda3.out‚ 1007.0625 disk.rbd1.out‚ 5.0126953125& disk.nvme0c0n1.out‚403.21337890625 disk.sda3.in‚0 disk.rbd1.in‚0# memory_percent‚84.7327470779419 disk.sda2.out‚0 network.recv‚398831$ disk.nvme0n1.out‚403.21337890625 disk.dm-0.in‚0 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 network.sent‚ 2.12477e+06 disk.sda1.out‚0 disk.dm-1.out‚ 1007.0625) proc.memory.availableMB‚ 2501.38671875 cpu‚23.75616765325191 disk.nvme0n1.in‚0 disk.drbd2266.out‚0! disk.dm-0.out‚403.21337890625 disk.drbd1716.in‚0) disk./.usagePercent‚15.64224364347629 disk.rbd0.in‚0 proc.memory.rssMB‚2819.625 disk.drbd2265.out‚0iI7*’­ ‚(:‡ –öâÔ×ÁÈ  gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚173.442' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.6884 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0sÃÃ8® ‚(:­ –öâÔÆ®á$ disk.nvme0n1.out‚422.48095703125) proc.memory.availableMB‚ 2501.38671875 proc.cpu.threads‚17 disk.sda3.out‚ 1017.61328125 disk.drbd2266.out‚0 proc.memory.rssMB‚2819.625 cpu‚23.781273893115323 disk.rbd1.in‚0* disk./.usagePercent‚15.642231099107113 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.dm-1.in‚0! disk.dm-0.out‚422.48095703125 disk.drbd2266.in‚0# memory_percent‚84.7327470779419 network.sent‚ 2.226919e+06 disk.sda3.in‚0 disk.rbd1.out‚ 5.154296875& disk.nvme0c0n1.out‚422.48095703125 disk.sda2.out‚0 disk.rbd0.in‚0 disk.sda.in‚0 network.recv‚404841 disk.drbd2265.in‚0 disk.dm-0.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.dm-1.out‚ 1017.61328125 disk.sda2.in‚0 disk.drbd2080.in‚0* proc.memory.percent‚17.209625244140625 disk.drbd2080.out‚0 disk.nvme0n1.in‚0 disk.sda1.out‚0 disk.nvme0c0n1.in‚0 disk.sda.out‚ 1017.61328125% disk./.usageGB‚233.08054733276367‘Lôm2¯ - _wandbruntime‚1045  _runtime‚1045'ÂkoÈ°  ' lr-AdamW/pg1‚0.00011623009595121846 ' lr-AdamW/pg2‚0.00011623009595121846 trainer/global_step‚1249 % _timestamp‚1.7883942725173573e+09 _runtime‚1045.497191405 _step‚98«âú0̱ Æ trainer/global_step‚1249 % _timestamp‚1.7883942725173573e+09 _runtime‚1045.497191405 _step‚98 ' lr-AdamW/pg1‚0.00011623009595121846 ' lr-AdamW/pg2‚0.00011623009595121846bnd‹2² - _wandbruntime‚1045  _runtime‚1045‘ó—ëº'³ ´O XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚69109576766464 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 918488528384 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6117141 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16681641 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10564033 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚67071091578880 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 913641745065 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 _runtime‚ 1045.50125825 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚40575625 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚70867271427584 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23031562 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 956746729472 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796096 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 956528933376 trainer/global_step‚1249 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚69910767397888 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚70027901409280 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚67071074539520 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6117605 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚40556098 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16681174 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23051087 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 918324642816 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10564036 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚69109593805824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚467 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚69910742494208 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 % _timestamp‚1.7883942725184517e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 918488528384 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚70027901409280 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 918324642816 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17524538 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚67984556212273 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6117141 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16681641 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10564033 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚67071091578880 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 913641745065 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 _runtime‚ 1045.50125825 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚40575625 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚70867271427584 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23031562 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 956746729472 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796096 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 956528933376 trainer/global_step‚1249 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚69910767397888 \ODeviceStatsMonitor.on_train_batch_¸‘ start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚70027901409280 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚67071074539520 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6117605 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚40556098 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16681174 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23051087 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 918324642816 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10564036 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚69109593805824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚467 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚69910742494208 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 % _timestamp‚1.7883942725184517e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 918488528384 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚70027901409280 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 918324642816 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17524538 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚67984556212273 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 I8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚68037180895381 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40587586 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 919199288320 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚69162966688768 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 trainer/global_step‚1249 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23069105 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚69162983728128 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚70922023659520 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 957279336448 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16694210 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚70082002078208 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚67123009302528 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 914188632213 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16693756 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6122115 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚67122992263168 % _timestamp‚1.7883942732021394e+09 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17538034 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚69964769226752 _step‚100 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚70082183016448 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚454 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23049554 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚70922255860224 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17538034 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚70082183016448 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17538032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6121664 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 919035389440 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17538032 _runtime‚1046.182359554 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚40587586 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚40607139 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚69162983728128 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23069105 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚69162966688768 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23049554 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 919035389440 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10572092 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚69964744323072 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 919199288320 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40607139 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚451òg‰Ê&· ÄM B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6122115 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚67122992263168 % _timestamp‚1.7883942732021394e+09 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17538034 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚69964769226752 _step‚100 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚70082183016448 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚454 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23049554 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚70922255860224 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17538034 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚70082183016448 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17538032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6121664 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 919035389440 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17538032 _runtime‚1046.182359554 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚40587586 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚40607139 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚69162983728128 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23069105 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚69162966688768 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23049554 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 919035389440 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10572092 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚69964744323072 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 919199288320 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40607139 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚451 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 914348704733 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚68037180895381 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40587586 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 919199288320 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚69162966688768 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 trainer/global_step‚1249 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23069105 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚69162983728128 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚70922023659520 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 957279336448 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16694210 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚70082002078208 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚67123009302528 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 914188632213 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16693756 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032dgMÙ2¸ - _wandbruntime‚1046  _runtime‚1046} h›O¹ É %loss/total_loss‚67.45701547521584 2loss/SRqqt4/assignment_loss‚2.3340019132241165 .loss/FBt3/assignment_loss‚0.48136717200712 /loss/FRt1/detection_loss‚2.6247299790382383 /loss/FRt2/assignment_loss‚4.075471191888764 2loss/SRqqt2/assignment_loss‚2.6494752549110574 /loss/FRt4/detection_loss‚2.9786078184843063 1loss/SRqqt3/detection_loss‚1.7819373392065365 0loss/SRqqt4/detection_loss‚2.426357562294101 _step‚101 0loss/FRt1/assignment_loss‚3.6766920280456543 1loss/SRqqt1/detection_loss‚1.7269721789035335 .loss/FBt3/detection_loss‚3.069531217865322 .loss/FRt3/assignment_loss‚4.34088058994241 /loss/FRt4/assignment_loss‚4.331488834648598 1loss/SRqqt3/assignment_loss‚2.304460033973058 0loss/FBt2/assignment_loss‚0.2285755221438387 0loss/FBt4/assignment_loss‚0.5160497861484495 0loss/FBt1/assignment_loss‚0.3538588041985584 .loss/FRt2/detection_loss‚2.699500460887239 .loss/FRt3/detection_loss‚2.435764547896712 0loss/SRqqt2/detection_loss‚3.514819473662275 trainer/global_step‚1249 % _timestamp‚1.7883942732038271e+09 _runtime‚1046.183420296 -loss/FBt1/detection_loss‚5.40592043534402 /loss/FBt2/detection_loss‚1.3068958210411377 epoch‚2 2loss/SRqqt1/assignment_loss‚1.7962475476869897 -loss/FBt4/detection_loss‚10.3974099617738ÒgŽèSº Í /loss/FRt1/detection_loss‚2.6247299790382383 /loss/FRt2/assignment_loss‚4.075471191888764 2loss/SRqqt2/assignment_loss‚2.6494752549110574 /loss/FRt4/detection_loss‚2.9786078184843063 1loss/SRqqt3/detection_loss‚1.7819373392065365 0loss/SRqqt4/detection_loss‚2.426357562294101 _step‚101 0loss/FRt1/assignment_loss‚3.6766920280456543 1loss/SRqqt1/detection_loss‚1.7269721789035335 .loss/FBt3/detection_loss‚3.069531217865322 .loss/FRt3/assignment_loss‚4.34088058994241 /loss/FRt4/assignment_loss‚4.331488834648598 1loss/SRqqt3/assignment_loss‚2.304460033973058 0loss/FBt2/assignment_loss‚0.2285755221438387 0loss/FBt4/assignment_loss‚0.5160497861484495 0loss/FBt1/assignment_loss‚0.3538588041985584 .loss/FRt2/detection_loss‚2.699500460887239 .loss/FRt3/detection_loss‚2.435764547896712 0loss/SRqqt2/detection_loss‚3.514819473662275 trainer/global_step‚1249 % _timestamp‚1.7883942732038271e+09 _runtime‚1046.183420296 -loss/FBt1/detection_loss‚5.40592043534402 /loss/FBt2/detection_loss‚1.3068958210411377 epoch‚2 2loss/SRqqt1/assignment_loss‚1.7962475476869897 -loss/FBt4/detection_loss‚10.3974099617738 %loss/total_loss‚67.45701547521584 2loss/SRqqt4/assignment_loss‚2.3340019132241165 .loss/FBt3/assignment_loss‚0.48136717200712epq'Þ’» ‚(:‡ ¥öâÔð¬ú  gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚172.906' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.5812 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0j1§24¼ ‚(:© ¥öâÔ²ß  cpu‚23.80562713105502 network.recv‚408263 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda1.in‚0( proc.memory.availableMB‚ 2501.1015625! disk.dm-0.out‚525.68408203125 disk.sda.in‚0 disk.drbd2265.in‚0 proc.memory.rssMB‚2819.625 network.sent‚ 2.232186e+06% disk./.usageGB‚233.08054733276367 disk.sda3.out‚ 1030.5546875 disk.nvme0c0n1.in‚0 disk.sda2.in‚0 disk.sda.out‚ 1030.5546875 disk.drbd2265.out‚0* disk./.usagePercent‚15.642231099107113 disk.rbd1.out‚ 5.240234375 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.dm-1.out‚ 1030.5546875$ disk.nvme0n1.out‚525.68408203125$ memory_percent‚84.73448753356934* proc.memory.percent‚17.209625244140625 proc.cpu.threads‚17 disk.rbd1.in‚0& disk.nvme0c0n1.out‚525.68408203125 disk.dm-0.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.sda1.out‚0 disk.sda3.in‚0 disk.rbd0.in‚0 disk.dm-1.in‚0âIS[’½ ‚(:‡ ´öâÔÈ” gpu.0.gpu‚57.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚172.906' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.5812 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Xó·ž.¾ ‚(:£ ´öâÔÕÚ* disk./.usagePercent‚15.642227002986564 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0# memory_percent‚85.4654312133789% disk./.usageGB‚233.08048629760742 disk.rbd1.out‚ 5.302734375 disk.nvme0c0n1.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.rbd1.in‚0& disk.nvme0c0n1.out‚531.85009765625 disk.sda.out‚ 1046.203125 disk.drbd2265.in‚0 disk.drbd2080.in‚0$ disk.nvme0n1.out‚531.85009765625 disk.sda.in‚0& proc.memory.availableMB‚ 2381.34375 disk.dm-0.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 proc.memory.rssMB‚2819.625* proc.memory.percent‚17.209625244140625 cpu‚23.829694169883183 proc.cpu.threads‚17 network.recv‚414270 disk.dm-1.in‚0! disk.dm-0.out‚531.85009765625 network.sent‚ 2.31952e+06 disk.dm-1.out‚ 1046.203125 disk.sda3.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚0 disk.drbd2266.out‚0 disk.sda3.out‚ 1046.203125 disk.drbd2080.out‚0Iòbö’¿ ‚(:‡ ÃöâÔÌžô  gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚176.679' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.3358 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¬ÈUÀ ‚(:Ê ÃöâÔø²Š proc.memory.rssMB‚2819.625 disk.sda.out‚ 1062.68359375 disk.drbd2266.out‚0 network.recv‚418987" disk.dm-0.out‚2497.41943359375 disk.rbd0.out‚0 disk.sda.in‚0* disk./.usagePercent‚15.832757794262022 disk.drbd2080.in‚0 disk.rbd0.in‚0% disk.nvme0n1.out‚2497.41943359375 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.nvme0c0n1.in‚ 4.05859375 network.sent‚ 2.325314e+06 disk.dm-0.in‚ 4.05859375 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda3.in‚0 cpu‚23.853255856757325 disk.drbd1716.out‚0' disk.nvme0c0n1.out‚2497.41943359375$ memory_percent‚85.46547889709473$ disk./.usageGB‚235.9195327758789 disk.drbd1716.in‚0 disk.drbd2266.in‚0 disk.dm-1.out‚ 1062.68359375 disk.sda3.out‚ 1062.68359375 disk.rbd1.in‚0 disk.rbd1.out‚ 5.302734375* proc.memory.percent‚17.209625244140625 disk.sda2.in‚0 disk.nvme0n1.in‚ 4.05859375 proc.cpu.threads‚17 disk.sda2.out‚0 disk.drbd2080.out‚0 disk.drbd2265.out‚0( proc.memory.availableMB‚ 2381.3359375Êp½J2Á - _wandbruntime‚1083  _runtime‚1083ÌAYHÉ Ã % _timestamp‚1.7883943106286232e+09 _runtime‚1083.608946288 _step‚102 ' lr-AdamW/pg1‚0.00011557345710488903 ' lr-AdamW/pg2‚0.00011557345710488903 trainer/global_step‚1299á=Íà Ç ' lr-AdamW/pg2‚0.00011557345710488903 trainer/global_step‚1299 % _timestamp‚1.7883943106286232e+09 _runtime‚1083.608946288 _step‚102 ' lr-AdamW/pg1‚0.00011557345710488903fVé•2Ä - _wandbruntime‚1083  _runtime‚1083n·Ó`xÅ µO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚474 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 953889802240 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚70615994651353 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18199338 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 954053687808 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18199338 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6352652 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73604844406784 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚72732962683904 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _step‚103 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚23954039 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17319164 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚471 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 993985565696 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17319638 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚42153377 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23934514 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10966986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚72610858841088 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚70615817539681 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚42133850 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 _runtime‚1083.609420343 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚471 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 993985565696 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17319638 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚42153377 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23934514 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10966986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚72610858841088 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚70615817539681 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚42133850 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 _runtime‚1083.609420343 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚474 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 953889802240 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚70615994651353 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18199338 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 954053687808 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18199338 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6352652 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73604844406784 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚72732962683904 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 _step‚103g­¯ 2Ç - _wandbruntime‚1084  _runtime‚1084è»gÅ&È ¿M QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚71832462803968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚460 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚73659593381376 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚70668442222789 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 954764447744 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 954600548864 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18212832 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23972057 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17331888 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚72664835766272 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 994523317248 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 949723849741 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 $ _timestamp‚1.788394311303151e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73659359083520 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚72664860669952 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚71832479843328 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 954764447744 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _runtime‚1084.283382188 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23972057 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6357306 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚69718878445568 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚72787063352832 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚70668619334669 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17332351 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10975045 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚42165338 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 954600548864 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚69718895484928 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚71832462803968 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚104 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚71832479843328 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10975042 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 994732711424 trainer/global_step‚1299 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18212832 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚42184891 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚104 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚71832479843328 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10975042 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 994732711424 trainer/global_step‚1299 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18212832 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚42184891 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17331888 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚72664835766272 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 994523317248 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 949723849741 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 $ _timestamp‚1.788394311303151e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73659359083520 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚72664860669952 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚71832479843328 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 954764447744 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _runtime‚1084.283382188 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23972057 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6357306 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚69718878445568 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚72787063352832 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚70668619334669 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17332351 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10975045 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚42165338 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 954600548864 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚69718895484928 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚71832462803968 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 FDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚490 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1031171085824 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18874136 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74448586036224 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚73247078867089 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24837466 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408320 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚76341884961792 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚72262846904320 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11369936 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚107 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚76342121370112 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18874138 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17958891 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚75438023958528 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚72262863943680 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚493 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75438023958528 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 989618847232 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 989454961664 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 984392035081 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43711602 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 984231962769 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75310950284288 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1030959581184 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 _runtime‚1121.963344862 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚75438204883456 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚24856991 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43731129 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211504640Á]Mw$Ö ¼O L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18874136 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74448586036224 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚73247078867089 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24837466 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408320 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚76341884961792 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚72262846904320 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11369936 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚107 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚76342121370112 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18874138 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17958891 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚75438023958528 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚72262863943680 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚493 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75438023958528 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 989618847232 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 989454961664 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 984392035081 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43711602 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 984231962769 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75310950284288 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1030959581184 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 _runtime‚1121.963344862 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚75438204883456 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚24856991 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43731129 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 211504640 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚74448586036224 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚75438204883456 E=DeviceStats÷’ØîKMonitor.on_train_batch_start/segment.all.allocated‚623 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚490 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1031171085824 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0kù§cx2× - _wandbruntime‚1122  _runtime‚1122Ñ¥émÇ&Ø ÁM UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18887632 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17971568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1031912198144 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1031704901120 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43743090 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6593573 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24875009 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75492124627456 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73299880662077 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 984938922229 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11377995 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43762643 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75492305565696 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24855458 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72314764627968 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 $ _timestamp‚1.788394349660119e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 990165708288 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6593103 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73299703550197 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18887632 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75492305565696 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72314781667328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚1122.640328618 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18887634 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 990329607168 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚74501975958528 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11377992 _step‚108 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚74501958919168 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43743090 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 985098994749 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 990165708288 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43762643 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚76396632110592 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚75364952113152 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24875009 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1349 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 990329607168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17971095 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0™•4Ë&Ù ÅM HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72314764627968 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 $ _timestamp‚1.788394349660119e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 990165708288 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6593103 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73299703550197 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18887632 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75492305565696 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72314781667328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚1122.640328618 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18887634 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 990329607168 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚74501975958528 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11377992 _step‚108 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚74501958919168 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43743090 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 985098994749 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 990165708288 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43762643 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚76396632110592 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚75364952113152 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24875009 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1349 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 990329607168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17971095 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18887632 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17971568 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1031912198144 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1031704901120 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43743090 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6593573 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24875009 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75492124627456 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73299880662077 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 984938922229 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11377995 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43762643 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75492305565696 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24855458 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0l{S/2Ú - _wandbruntime‚1122  _runtime‚1122°EL†VÛ Ð /loss/FBt3/detection_loss‚2.5038298714284752 2loss/SRqqt1/assignment_loss‚1.6118955387553173 0loss/FBt4/assignment_loss‚0.3486477656696134 /loss/FRt3/detection_loss‚2.5668234181675045 1loss/SRqqt3/detection_loss‚1.9420110761587108 .loss/FRt4/detection_loss‚2.521098802186022 .loss/FBt4/detection_loss‚4.721513214810141 &loss/total_loss‚59.277361554747905 epoch‚2 trainer/global_step‚1349 0loss/SRqqt2/detection_loss‚4.646411141114576 1loss/SRqqt4/assignment_loss‚2.410115427180622 0loss/FBt3/assignment_loss‚0.5340505626903636 _step‚109 /loss/FBt2/detection_loss‚1.1493250558069914 /loss/FRt1/assignment_loss‚3.369055880370893 0loss/FRt3/assignment_loss‚4.9354160879597515 0loss/FBt1/assignment_loss‚0.4896741636860367 0loss/SRqqt4/detection_loss‚2.344356240851156 _runtime‚1122.641393517 0loss/FRt2/assignment_loss‚3.1942366267505444 0loss/FBt2/assignment_loss‚0.3428087940202941 /loss/FBt1/detection_loss‚3.4509700157103085 1loss/SRqqt2/assignment_loss‚2.725510154451643 1loss/SRqqt3/assignment_loss‚2.174561977031685 .loss/FRt2/detection_loss‚2.695214309680619 /loss/FRt4/assignment_loss‚4.267339742544926 /loss/FRt1/detection_loss‚2.6148883876047635 1loss/SRqqt1/detection_loss‚1.7176073001169447 % _timestamp‚1.7883943496617813e+09”‚m7ZÜ Ô /loss/FBt2/detection_loss‚1.1493250558069914 /loss/FRt1/assignment_loss‚3.369055880370893 0loss/FRt3/assignment_loss‚4.9354160879597515 0loss/FBt1/assignment_loss‚0.4896741636860367 0loss/SRqqt4/detection_loss‚2.344356240851156 _runtime‚1122.641393517 0loss/FRt2/assignment_loss‚3.1942366267505444 0loss/FBt2/assignment_loss‚0.3428087940202941 /loss/FBt1/detection_loss‚3.4509700157103085 1loss/SRqqt2/assignment_loss‚2.725510154451643 1loss/SRqqt3/assignment_loss‚2.174561977031685 .loss/FRt2/detection_loss‚2.695214309680619 /loss/FRt4/assignment_loss‚4.267339742544926 /loss/FRt1/detection_loss‚2.6148883876047635 1loss/SRqqt1/detection_loss‚1.7176073001169447 % _timestamp‚1.7883943496617813e+09 /loss/FBt3/detection_loss‚2.5038298714284752 2loss/SRqqt1/assignment_loss‚1.6118955387553173 0loss/FBt4/assignment_loss‚0.3486477656696134 /loss/FRt3/detection_loss‚2.5668234181675045 1loss/SRqqt3/detection_loss‚1.9420110761587108 .loss/FRt4/detection_loss‚2.521098802186022 .loss/FBt4/detection_loss‚4.721513214810141 &loss/total_loss‚59.277361554747905 epoch‚2 trainer/global_step‚1349 0loss/SRqqt2/detection_loss‚4.646411141114576 1loss/SRqqt4/assignment_loss‚2.410115427180622 0loss/FBt3/assignment_loss‚0.5340505626903636 _step‚109mcµ(=Ý ‚(:’ ðöâԱᣠ gpu.0.gpu‚43.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚173.195' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.638999999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'ÿyOÞ ‚(:Ä ðöâÔßß°% disk.nvme0n1.out‚9384.09716796875 disk.sda1.in‚0) disk./.usagePercent‚16.22752678821877% disk./.usageGB‚241.80187606811523 disk.sda3.in‚0 disk.drbd2266.out‚0 proc.cpu.threads‚17" disk.nvme0c0n1.in‚ 167.08203125 disk.dm-0.in‚ 167.08203125 disk.nvme0n1.in‚ 167.08203125# memory_percent‚85.4672908782959( proc.memory.availableMB‚ 2381.0390625 network.recv‚432658 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.drbd2266.in‚0* proc.memory.percent‚17.209625244140625 disk.rbd1.out‚ 5.5068359375 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda.out‚ 1082.4375 disk.drbd2265.out‚0 disk.dm-1.out‚ 1082.4375 proc.memory.rssMB‚2819.625 network.sent‚ 2.423435e+06 disk.sda3.out‚ 1082.4375' disk.nvme0c0n1.out‚9384.09716796875 disk.drbd2080.out‚0 disk.sda.in‚0 disk.dm-1.in‚0 cpu‚23.919656140112693" disk.dm-0.out‚9384.09716796875^$”ëß ‚(:’ ÿöâÔ‘°  gpu.0.gpu‚52.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.842' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.168400000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Œ6©Mà ‚(: ÿöâÔ¯çÞ  disk.rbd0.in‚0 disk.dm-0.out‚9566.068359375& proc.memory.availableMB‚ 2380.96875 disk.drbd2265.in‚0 disk.dm-0.in‚ 167.265625 proc.memory.rssMB‚2819.625) disk./.usagePercent‚16.31473191464471 disk.drbd2265.out‚0$ memory_percent‚85.46772003173828$ disk./.usageGB‚243.1012954711914 disk.rbd1.in‚0 disk.rbd1.out‚ 5.5927734375 cpu‚23.940880758012902 disk.drbd2266.out‚0 disk.sda3.in‚0 disk.nvme0n1.in‚ 167.265625 network.recv‚439197 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.sda.out‚ 1107.13671875 proc.cpu.threads‚17 disk.sda2.in‚0 disk.drbd2080.out‚0# disk.nvme0n1.out‚9566.068359375 disk.sda.in‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.dm-1.in‚0 disk.nvme0c0n1.in‚ 167.265625 disk.rbd0.out‚0 disk.sda3.out‚ 1107.13671875% disk.nvme0c0n1.out‚9566.068359375* proc.memory.percent‚17.209625244140625 disk.drbd1716.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1107.13671875 network.sent‚ 2.510572e+06 disk.drbd2080.in‚0{¢Uá ‚(:’ Ž÷âÔ“ùÖ  gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚178.328' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚35.665600000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0}Ù»ÀKâ ‚(:À Ž÷âÔÖÂÝ  network.recv‚443030% disk./.usageGB‚243.06530380249023 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2266.out‚0( proc.memory.availableMB‚ 2381.4453125 proc.cpu.threads‚17 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 170.640625 disk.drbd2266.in‚0 disk.rbd1.out‚ 5.8505859375 disk.nvme0c0n1.in‚ 170.640625 cpu‚23.961703146255097 disk.sda2.out‚0 disk.rbd0.in‚0 disk.sda.in‚0 disk.sda3.in‚0 disk.rbd0.out‚0 disk.drbd2265.in‚0 disk.sda3.out‚ 1127.515625$ memory_percent‚85.46481132507324 proc.memory.rssMB‚2819.625* disk./.usagePercent‚16.312316483559155 disk.sda.out‚ 1127.515625 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2265.out‚0* proc.memory.percent‚17.209625244140625 network.sent‚ 2.515962e+06 disk.sda1.out‚0 disk.dm-1.out‚ 1127.515625% disk.nvme0c0n1.out‚10981.73828125 disk.sda2.in‚0# disk.nvme0n1.out‚10981.73828125 disk.sda1.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 170.640625 disk.dm-0.out‚10981.73828125¯¥t2ã - _wandbruntime‚1160  _runtime‚1160üݘGÉä à ' lr-AdamW/pg1‚0.00011412202091286283 ' lr-AdamW/pg2‚0.00011412202091286283 trainer/global_step‚1399 % _timestamp‚1.7883943871873026e+09 _runtime‚1160.167228721 _step‚110S±ÄÍå Ç ' lr-AdamW/pg2‚0.00011412202091286283 trainer/global_step‚1399 % _timestamp‚1.7883943871873026e+09 _runtime‚1160.167228721 _step‚110 ' lr-AdamW/pg1‚0.00011412202091286283n3<ß×2æ - _wandbruntime‚1160  _runtime‚1160÷„Ùç ¾O PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25759943 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 232214016 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚79079230440448 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚77118082151424 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚78011041727488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚74858750126080 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚74858733086720 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 % _timestamp‚1.7883943871884186e+09 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚79079462654464 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25740418 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 207310336 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚475 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19548936 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚75878340194497 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1019607107777 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚45308881 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚77118065112064 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚75878517306169 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1025184006656 QBDeviceStatsMonitor.on_traÇáKuëin_batch_start/reserved_bytes.all.current‚ 9575596032 trainer/global_step‚1399 _step‚111 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45289354 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚78143266158080 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚78143266158080 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11772883 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚77118065112064 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25759943 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1068420926976 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19548938 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚78011016823808 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚472 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚78143085233152 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚25740418 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1025020121088 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18596646 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11772886 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19548938 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚78143085233152 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1025184006656 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1019767180089 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928ºÈ'è ÂO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11772883 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚77118065112064 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25759943 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1068420926976 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19548938 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚78011016823808 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚472 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚78143085233152 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚25740418 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1025020121088 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18596646 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11772886 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19548938 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚78143085233152 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1025184006656 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1019767180089 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25759943 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 232214016 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚79079230440448 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚77118082151424 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚78011041727488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚74858750126080 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚74858733086720 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 % _timestamp‚1.7883943871884186e+09 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚79079462654464 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25740418 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 207310336 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚475 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19548936 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚75878340194497 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1019607107777 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚45308881 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚77118065112064 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚75878517306169 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1025184006656 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 trainer/global_step‚1399 _step‚111 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45289354 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚78143266158080 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚78143266158080 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207oÖRl2é - _wandbruntime‚1160  _runtime‚1160#‡b²Î&ê ÈM KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19562432 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1020474139757 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25758410 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚25758410 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚79134209003520 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚78197366840320 _runtime‚1160.843802703 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚78197185902080 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚45340395 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19562434 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚457 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚77171455034368 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45320842 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45340395 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6829071 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25777961 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18609556 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11780942 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚78065043556352 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25777961 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18610016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883943878635979e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19562434 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6828614 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75931141989485 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19562432 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1069165447168 _step‚112 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1025730867712 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚78197366840320 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75930964877605 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚77171472073728 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚74910667849728 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚78065018652672 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 4341104646fCR*ë ÌM D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7883943878635979e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19562434 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6828614 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75931141989485 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19562432 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1069165447168 _step‚112 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1025730867712 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚78197366840320 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75930964877605 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚77171472073728 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚74910667849728 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚78065018652672 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19562432 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1020474139757 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25758410 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚25758410 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚79134209003520 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚78197366840320 _runtime‚1160.843802703 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚78197185902080 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚45340395 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19562434 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.sma‡ZRp¨ ll_pool.current‚457 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚77171455034368 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45320842 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45340395 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6829071 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25777961 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18609556 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11780942 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚78065043556352 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25777961 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18610016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0pJ ù³2ì - _wandbruntime‚1160  _runtime‚1160ƒ •ÊUí Ï 2loss/SRqqt4/assignment_loss‚2.1724754245842206 0loss/FBt4/assignment_loss‚0.3219726071322878 .loss/FRt1/detection_loss‚2.502374893724918 /loss/FRt2/assignment_loss‚4.266904196361216 1loss/FBt1/assignment_loss‚0.48896436767656426 /loss/FRt3/detection_loss‚2.4299111374910325 1loss/SRqqt3/detection_loss‚1.7195640769846077 _step‚113 0loss/SRqqt4/detection_loss‚2.414277187165092 _runtime‚1160.844935911 2loss/SRqqt3/assignment_loss‚2.0581707382881187 1loss/FBt2/assignment_loss‚0.19926419551950203 .loss/FBt1/detection_loss‚5.533951973137648 .loss/FBt2/detection_loss‚1.768660320355181 %loss/total_loss‚64.17624209665749 trainer/global_step‚1399 /loss/FRt4/assignment_loss‚4.590572002141372 0loss/FBt3/assignment_loss‚0.5690543633466839 epoch‚2 .loss/FBt4/detection_loss‚5.394412305044091 2loss/SRqqt1/assignment_loss‚1.9393137113530508 /loss/FRt4/detection_loss‚2.6650435940726944 1loss/SRqqt1/detection_loss‚1.7751830204917016 .loss/FBt3/detection_loss‚3.228083705534274 0loss/FRt1/assignment_loss‚3.8354120123386384 1loss/SRqqt2/assignment_loss‚3.064865433808529 0loss/SRqqt2/detection_loss‚4.051446137103167 /loss/FRt3/assignment_loss‚4.576983367187389 .loss/FRt2/detection_loss‚2.609381325815509 % _timestamp‚1.7883943878653321e+09ØÈêuYî Ó /loss/FRt3/assignment_loss‚4.576983367187389 .loss/FRt2/detection_loss‚2.609381325815509 % _timestamp‚1.7883943878653321e+09 2loss/SRqqt4/assignment_loss‚2.1724754245842206 0loss/FBt4/assignment_loss‚0.3219726071322878 .loss/FRt1/detection_loss‚2.502374893724918 /loss/FRt2/assignment_loss‚4.266904196361216 1loss/FBt1/assignment_loss‚0.48896436767656426 /loss/FRt3/detection_loss‚2.4299111374910325 1loss/SRqqt3/detection_loss‚1.7195640769846077 _step‚113 0loss/SRqqt4/detection_loss‚2.414277187165092 _runtime‚1160.844935911 2loss/SRqqt3/assignment_loss‚2.0581707382881187 1loss/FBt2/assignment_loss‚0.19926419551950203 .loss/FBt1/detection_loss‚5.533951973137648 .loss/FBt2/detection_loss‚1.768660320355181 %loss/total_loss‚64.17624209665749 trainer/global_step‚1399 /loss/FRt4/assignment_loss‚4.590572002141372 0loss/FBt3/assignment_loss‚0.5690543633466839 epoch‚2 .loss/FBt4/detection_loss‚5.394412305044091 2loss/SRqqt1/assignment_loss‚1.9393137113530508 /loss/FRt4/detection_loss‚2.6650435940726944 1loss/SRqqt1/detection_loss‚1.7751830204917016 .loss/FBt3/detection_loss‚3.228083705534274 0loss/FRt1/assignment_loss‚3.8354120123386384 1loss/SRqqt2/assignment_loss‚3.064865433808529 0loss/SRqqt2/detection_loss‚4.051446137103167q PÁ’ï ‚(:‡ ÷âÔñØÏ  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.686' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.1372 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Èq«gBð ‚(:· ÷âÔ툺 disk.rbd0.in‚0 disk.nvme0n1.in‚171.25 disk.sda3.out‚ 1143.08203125 disk.dm-0.out‚ 10987.8984375* proc.memory.percent‚17.209625244140625 network.recv‚446743 disk.sda.out‚ 1143.08203125 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.dm-1.out‚ 1143.08203125 disk.sda3.in‚0 disk.rbd1.in‚0" disk.nvme0n1.out‚ 10987.8984375 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 cpu‚23.981826677583125) disk./.usagePercent‚16.31191583176807% disk./.usageGB‚243.05933380126953 disk.rbd0.out‚0 disk.nvme0c0n1.in‚171.25 disk.drbd1716.out‚0$ memory_percent‚85.46669483184814 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.sda1.out‚0$ disk.nvme0c0n1.out‚ 10987.8984375 proc.memory.rssMB‚2819.625 network.sent‚ 2.521427e+06 disk.sda.in‚0 disk.sda2.out‚0 disk.drbd2080.out‚0) proc.memory.availableMB‚ 2381.13671875 proc.cpu.threads‚17 disk.rbd1.out‚ 5.8505859375 disk.dm-0.in‚171.25]3âWñ ‚(:’ ¬÷âÔÂœ¬  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚170.058' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.011599999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)ÊKò ‚(:À ¬÷âÔ—¬ disk.sda2.out‚0 disk.drbd1716.out‚0 disk.sda.out‚ 1160.80859375 disk.drbd2266.out‚0 disk.dm-0.in‚171.25" disk.dm-0.out‚10988.7529296875( proc.memory.availableMB‚ 2381.0859375% disk.nvme0n1.out‚10988.7529296875 disk.sda1.out‚0 disk.nvme0c0n1.in‚171.25$ memory_percent‚85.46700477600098 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0' disk.nvme0c0n1.out‚10988.7529296875* proc.memory.percent‚17.209625244140625 cpu‚24.001789308509338% disk./.usageGB‚243.05878067016602 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda3.in‚0 disk.rbd1.out‚ 5.9130859375 proc.memory.rssMB‚2819.625 disk.rbd0.out‚0 disk.nvme0n1.in‚171.25 network.sent‚ 2.608842e+06 disk.drbd2080.in‚0 disk.sda.in‚0 disk.dm-1.out‚ 1160.80859375 proc.cpu.threads‚17 disk.rbd1.in‚0 network.recv‚453067* disk./.usagePercent‚16.311878710675604 disk.drbd2265.in‚0 disk.sda3.out‚ 1160.80859375³´FÈ2ó - _wandbruntime‚1198  _runtime‚1198Œ0T Éô à ' lr-AdamW/pg2‚0.00011332843541638316 trainer/global_step‚1449 % _timestamp‚1.7883944254013834e+09 _runtime‚1198.381192934 _step‚114 ' lr-AdamW/pg1‚0.00011332843541638316+ ¨ßÍõ Ç trainer/global_step‚1449 % _timestamp‚1.7883944254013834e+09 _runtime‚1198.381192934 _step‚114 ' lr-AdamW/pg1‚0.00011332843541638316 ' lr-AdamW/pg2‚0.00011332843541638316r/-í2ö - _wandbruntime‚1198  _runtime‚1198Êv•Ã'÷ ½O PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20223738 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚79787561227264 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1105423513088 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚26662895 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80848146507776 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚79787578266624 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80711133170688 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚80711108267008 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26643370 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 trainer/global_step‚1449 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12175833 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1060749166080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1060749166080 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚469 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚78509778633577 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚79787561227264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20223736 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80848146507776 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚78509601521905 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12175836 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1105632920576 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20223738 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚81816531780096 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚80848327432704 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚26643370 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20223736 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1060585280512 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1060585280512 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7058469 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚79787578266624 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19234774 $ _timestamp‚1.788394425402439e+09 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26662895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80848146507776 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚79787578266624 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80711133170688 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚80711108267008 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26643370 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 trainer/global_step‚1449 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12175833 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1060749166080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1060749166080 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚469 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚78509778633577 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚79787561227264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20223736 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80848146507776 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚78509601521905 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12175836 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1105632920576 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20223738 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚81816531780096 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚80848327432704 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚26643370 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20223736 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1060585280512 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1060585280512 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7058469 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚79787578266624 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19234774 $ _timestamp‚1.788394425402439e+09 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26662895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚26680913 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚80765110095872 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1061296027136 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚77506536992768 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1055849284765 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 957559603 Qô”72 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12183895 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19247510 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19247051 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7063615 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77506554032128 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46918147 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80902247176704 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1061459926016 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚81871515313664 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46918147 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7063159 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20237234 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚46898594 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238492160 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1106380314112 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1106166725632 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _runtime‚1199.063820077 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80902428114944 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 trainer/global_step‚1449 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1055689212245 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26661362 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26680913 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20237234 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚78562226205013 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 _step‚116þ|FÑ&û ËM FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46918147 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80902247176704 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1061459926016 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚81871515313664 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46918147 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7063159 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20237234 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚46898594 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238492160 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1106380314112 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1106166725632 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _runtime‚1199.063820077 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80902428114944 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 trainer/global_step‚1449 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1055689212245 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26661362 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26680913 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20237234 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚78562226205013 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 _step‚116 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚80765134999552 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚78562403316893 $ _timestamp‚1.788394426083569e+09 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20237232 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1061459926016 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1061296027136 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20237232 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚26680913 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚80765110095872 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1061296027136 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚77506536992768 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1055849284765 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12183895 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19247510 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19247051 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7063615 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77506554032128 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0t†«a2ü - _wandbruntime‚1199  _runtime‚1199þ8<$Vý Ð /loss/FRt4/assignment_loss‚4.304111072830126 0loss/FBt1/assignment_loss‚0.6457832489390191 1loss/FBt4/assignment_loss‚0.40622307313003997 0loss/SRqqt4/detection_loss‚2.582831363852431 .loss/FBt4/detection_loss‚4.409927672353284 $ _timestamp‚1.788394426085353e+09 /loss/FRt2/assignment_loss‚4.121236468763913 /loss/FRt3/assignment_loss‚4.073055591848162 0loss/FBt3/assignment_loss‚0.4927348151422848 /loss/FRt2/detection_loss‚2.4231841078575918 .loss/FRt3/detection_loss‚2.600077552927865 2loss/SRqqt3/assignment_loss‚1.8349936865583187 1loss/SRqqt2/detection_loss‚3.1282149955283765 /loss/FBt1/detection_loss‚3.3217423914409263 trainer/global_step‚1449 2loss/SRqqt1/assignment_loss‚1.9777942947333766 _step‚117 0loss/SRqqt3/detection_loss‚2.144635776511761 /loss/FBt2/detection_loss‚1.0317440320837015 2loss/SRqqt2/assignment_loss‚3.2549080848693848 1loss/FBt2/assignment_loss‚0.27979076683812365 1loss/SRqqt1/detection_loss‚1.7346170969780474 1loss/SRqqt4/assignment_loss‚2.250233332921819 /loss/FRt1/detection_loss‚2.4060103873599252 _runtime‚ 1199.06499383 /loss/FRt1/assignment_loss‚3.904509278963197 /loss/FRt4/detection_loss‚2.4597206553992104 .loss/FBt3/detection_loss‚5.284368321389801 &loss/total_loss‚61.072448069220684 epoch‚2´´£ØZþ Ô 0loss/SRqqt3/detection_loss‚2.144635776511761 /loss/FBt2/detection_loss‚1.0317440320837015 2loss/SRqqt2/assignment_loss‚3.2549080848693848 1loss/FBt2/assignment_loss‚0.27979076683812365 1loss/SRqqt1/detection_loss‚1.7346170969780474 1loss/SRqqt4/assignment_loss‚2.250233332921819 /loss/FRt1/detection_loss‚2.4060103873599252 _runtime‚ 1199.06499383 /loss/FRt1/assignment_loss‚3.904509278963197 /loss/FRt4/detection_loss‚2.4597206553992104 .loss/FBt3/detection_loss‚5.284368321389801 &loss/total_loss‚61.072448069220684 epoch‚2 /loss/FRt4/assignment_loss‚4.304111072830126 0loss/FBt1/assignment_loss‚0.6457832489390191 1loss/FBt4/assignment_loss‚0.40622307313003997 0loss/SRqqt4/detection_loss‚2.582831363852431 .loss/FBt4/detection_loss‚4.409927672353284 $ _timestamp‚1.788394426085353e+09 /loss/FRt2/assignment_loss‚4.121236468763913 /loss/FRt3/assignment_loss‚4.073055591848162 0loss/FBt3/assignment_loss‚0.4927348151422848 /loss/FRt2/detection_loss‚2.4231841078575918 .loss/FRt3/detection_loss‚2.600077552927865 2loss/SRqqt3/assignment_loss‚1.8349936865583187 1loss/SRqqt2/detection_loss‚3.1282149955283765 /loss/FBt1/detection_loss‚3.3217423914409263 trainer/global_step‚1449 2loss/SRqqt1/assignment_loss‚1.9777942947333766 _step‚117u/\’ÿ ‚(:‡ »÷âÔÂŠ×  gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚177.792' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.5584 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ýâM–@€ ‚(:µ »÷âÔ»™§ network.sent‚ 2.615009e+06% disk./.usageGB‚243.05892181396484 disk.dm-0.in‚171.25 disk.rbd0.in‚0 disk.sda.out‚1175.625 disk.dm-1.in‚0 disk.sda3.in‚0( proc.memory.availableMB‚ 2380.7890625 disk.drbd1716.out‚0 disk.rbd1.in‚0" disk.dm-0.out‚10988.9326171875 disk.sda2.in‚0# proc.memory.rssMB‚ 2819.62890625 cpu‚24.020888373042826 disk.sda2.out‚0 disk.drbd2080.out‚0 disk.sda3.out‚1175.625$ memory_percent‚85.46881675720215 network.recv‚457614 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.drbd1716.in‚0 disk.sda.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.dm-1.out‚1175.625 disk.rbd1.out‚ 5.9130859375 disk.nvme0c0n1.in‚171.25' disk.nvme0c0n1.out‚10988.9326171875 disk.drbd2080.in‚0% disk.nvme0n1.out‚10988.9326171875 disk.drbd2266.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚171.25) disk./.usagePercent‚16.31188818295437 disk.sda1.in‚0* proc.memory.percent‚17.209649085998535 proc.cpu.threads‚17ç-üK’ ‚(:‡ Ê÷âÔ¼øº  gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚169.422' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.8844 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒýžeH‚ ‚(:½ Ê÷âÔ–å„ disk.sda.in‚0 disk.rbd0.in‚0 disk.sda3.in‚0 disk.rbd1.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0# proc.memory.rssMB‚ 2819.62890625 network.sent‚ 2.701495e+06 disk.drbd2265.in‚0( disk.nvme0c0n1.out‚10991.87841796875 disk.sda2.in‚0 disk.nvme0n1.in‚171.25& disk.nvme0n1.out‚10991.87841796875 disk.sda1.in‚0# disk.dm-0.out‚10991.87841796875' proc.memory.availableMB‚ 2381.359375* proc.memory.percent‚17.209649085998535* disk./.usagePercent‚16.311919927888617% disk./.usageGB‚243.05939483642578 disk.sda3.out‚ 1196.640625 disk.dm-0.in‚171.25 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 cpu‚24.03990399602037 disk.drbd2266.out‚0 disk.dm-1.out‚ 1196.640625$ memory_percent‚85.46533584594727 disk.nvme0c0n1.in‚171.25 disk.sda2.out‚0 disk.rbd0.out‚0 disk.sda.out‚ 1196.640625 proc.cpu.threads‚17 network.recv‚462923 disk.sda1.out‚0 disk.rbd1.out‚ 6.0546875äƒg’ƒ ‚(:‡ Ù÷âÔ´¹  gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚171.841' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.3682 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0›0¸SN„ ‚(:à Ù÷âÔßÆð  proc.cpu.threads‚17 disk.sda1.in‚0 disk.sda3.in‚0 disk.rbd1.in‚0 cpu‚24.05802155490155 network.sent‚ 2.706953e+06 disk.sda1.out‚0 disk.sda2.in‚0) disk./.usagePercent‚16.31193528834067% disk./.usageGB‚243.05962371826172 disk.nvme0n1.in‚171.25 disk.sda2.out‚0* proc.memory.percent‚17.209672927856445 disk.dm-0.in‚171.25 disk.drbd2080.in‚0 disk.drbd2080.out‚0& disk.nvme0n1.out‚10992.03857421875 disk.dm-1.in‚0 disk.nvme0c0n1.in‚171.25 disk.drbd1716.in‚0 disk.rbd1.out‚ 6.0546875 network.recv‚466420 disk.sda.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.dm-1.out‚ 1221.08984375) proc.memory.availableMB‚ 2380.62109375 disk.rbd0.out‚0( disk.nvme0c0n1.out‚10992.03857421875 disk.rbd0.in‚0 disk.drbd2266.out‚0$ memory_percent‚85.46984195709229" proc.memory.rssMB‚ 2819.6328125# disk.dm-0.out‚10992.03857421875 disk.sda.out‚ 1221.08984375 disk.drbd2265.out‚0 disk.sda3.out‚ 1221.08984375 disk.drbd1716.out‚0–[{|2… - _wandbruntime‚1237  _runtime‚1237ŽšƒÇ† Á & lr-AdamW/pg1‚0.0001124904759043877 & lr-AdamW/pg2‚0.0001124904759043877 trainer/global_step‚1499 % _timestamp‚1.7883944641800768e+09 _runtime‚1237.159869349 _step‚118 êˇ Å _step‚118 & lr-AdamW/pg1‚0.0001124904759043877 & lr-AdamW/pg2‚0.0001124904759043877 trainer/global_step‚1499 % _timestamp‚1.7883944641800768e+09 _runtime‚1237.159869349v¯72ˆ - _wandbruntime‚1237  _runtime‚1237£Ð%$‰ ¾O PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20898536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚83553388707328 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚82457057342464 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚80050505451520 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1142816782336 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚48444858 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚82457074381824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1096150439936 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚82457057342464 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚84554041396224 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20898538 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27565847 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚83553388707328 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚83553207782400 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1096150439936 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚83411199710208 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 _step‚119 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚83411224613888 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚83553207782400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1096314325504 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1096314325504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12578783 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1090517470105 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚80050522490880 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_byXÞý  tes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7293683 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602624 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19872003 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698944 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚27565847 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚82457074381824 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _runtime‚1237.164241407 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚463 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 % _timestamp‚1.7883944641812458e+09 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19872469 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚1499õ;lÄÈ'Š ÂO _step‚119 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚83411224613888 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚83553207782400 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1096314325504 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1096314325504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12578783 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1090517470105 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚80050522490880 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7293683 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602624 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19872003 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698944 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚27565847 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚82457074381824 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 _runtime‚1237.164241407 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚463 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 % _timestamp‚1.7883944641812458e+09 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19872469 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 trainer/global_step‚1499 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20898536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚83553388707328 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚82457057342464 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚80050505451520 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1142816782336 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚48444858 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚82457074381824 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1096150439936 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚82457057342464 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚84554041396224 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20898538 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27565847 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚83553388707328 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚83553207782400 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1096150439936 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚83411199710208 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0w`Ø®H2‹ - _wandbruntime‚1237  _runtime‚1237¸†K?Î&Œ ÈM REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12586842 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚80102423175168 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48495899 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20912032 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19884698 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1091224429773 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1097025085440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚82510464304128 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚82510447264768 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚48476346 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚83607489389568 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 _runtime‚1237.843000425 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚82510464304128 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1143556944384 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1091064357253 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27583865 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27583865 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚81193487532421 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚83607308451328 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12586845 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 trainer/global_step‚1499 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48495899 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27564314 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7298329 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚473 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚27564314 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20912034 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚82510447264768 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚476 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚83465226442752 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚81193664644301 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚80102440214528 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20912032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19885174 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7297856 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513x ³’u& ÌM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1097025085440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚82510464304128 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚82510447264768 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚48476346 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚83607489389568 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 _runtime‚1237.843000425 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚82510464304128 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1143556944384 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1091064357253 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27583865 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27583865 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚81193487532421 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚83607308451328 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12586845 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 trainer/global_step‚1499 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48495899 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27564314 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7298329 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚473 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚27564314 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20912034 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚82510447264768 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚476 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚83465226442752 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚81193664644301 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚80102440214528 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20912032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19885174 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7297856 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12586842 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚80102423175168 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48495899 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20912032 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19884698 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 10912244297e‰Sˆ]73 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464xüA—2Ž - _wandbruntime‚1237  _runtime‚1237šô›U Ï 0loss/SRqqt2/detection_loss‚3.388786700036791 1loss/SRqqt3/assignment_loss‚2.000642288927779 0loss/FBt1/assignment_loss‚0.4059982981666205 1loss/FBt4/assignment_loss‚0.30989018120382517 /loss/FRt2/detection_loss‚2.5371193957549556 .loss/FRt4/detection_loss‚2.377331574665052 .loss/FBt3/detection_loss‚3.168921780997309 .loss/FRt1/detection_loss‚2.139555864223913 .loss/FRt3/detection_loss‚2.180677993044461 .loss/FBt4/detection_loss‚4.967570444811946 _step‚121 1loss/SRqqt1/detection_loss‚1.6168933586917655 0loss/SRqqt4/detection_loss‚2.565968577563763 /loss/FBt2/detection_loss‚0.9562803198066326 0loss/FRt2/assignment_loss‚3.7837403026627907 /loss/FRt3/assignment_loss‚4.419954851882099 /loss/FRt4/assignment_loss‚4.682572804878805 1loss/SRqqt3/detection_loss‚1.9070948732057305 % _timestamp‚1.7883944648644605e+09 0loss/FRt1/assignment_loss‚3.7270466590128026 2loss/SRqqt1/assignment_loss‚1.8323135191071642 2loss/SRqqt4/assignment_loss‚2.2675309071317313 epoch‚2 _runtime‚ 1237.84406314 2loss/SRqqt2/assignment_loss‚2.7606881260871887 trainer/global_step‚1499 .loss/FBt1/detection_loss‚4.064996202786763 %loss/total_loss‚58.66772833209133 0loss/FBt2/assignment_loss‚0.2694276431208964 1loss/FBt3/assignment_loss‚0.33672566432054235@/*ýY Ó 0loss/FBt2/assignment_loss‚0.2694276431208964 1loss/FBt3/assignment_loss‚0.33672566432054235 0loss/SRqqt2/detection_loss‚3.388786700036791 1loss/SRqqt3/assignment_loss‚2.000642288927779 0loss/FBt1/assignment_loss‚0.4059982981666205 1loss/FBt4/assignment_loss‚0.30989018120382517 /loss/FRt2/detection_loss‚2.5371193957549556 .loss/FRt4/detection_loss‚2.377331574665052 .loss/FBt3/detection_loss‚3.168921780997309 .loss/FRt1/detection_loss‚2.139555864223913 .loss/FRt3/detection_loss‚2.180677993044461 .loss/FBt4/detection_loss‚4.967570444811946 _step‚121 1loss/SRqqt1/detection_loss‚1.6168933586917655 0loss/SRqqt4/detection_loss‚2.565968577563763 /loss/FBt2/detection_loss‚0.9562803198066326 0loss/FRt2/assignment_loss‚3.7837403026627907 /loss/FRt3/assignment_loss‚4.419954851882099 /loss/FRt4/assignment_loss‚4.682572804878805 1loss/SRqqt3/detection_loss‚1.9070948732057305 % _timestamp‚1.7883944648644605e+09 0loss/FRt1/assignment_loss‚3.7270466590128026 2loss/SRqqt1/assignment_loss‚1.8323135191071642 2loss/SRqqt4/assignment_loss‚2.2675309071317313 epoch‚2 _runtime‚ 1237.84406314 2loss/SRqqt2/assignment_loss‚2.7606881260871887 trainer/global_step‚1499 .loss/FBt1/detection_loss‚4.064996202786763 %loss/total_loss‚58.66772833209133yß÷íF‘ ‚(:» è÷âÔ— Ì  disk.dm-1.in‚0 disk.sda3.out‚ 1239.046875 disk.dm-0.in‚171.25# disk.dm-0.out‚10994.87255859375 disk.drbd1716.out‚0 disk.nvme0n1.in‚171.25% proc.memory.availableMB‚ 2380.8125 disk.sda2.in‚0& disk.nvme0n1.out‚10994.87255859375 disk.sda.in‚0 proc.cpu.threads‚17 disk.sda1.out‚0 disk.rbd1.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0% disk./.usageGB‚243.05960083007812 disk.rbd1.out‚ 6.1171875 network.recv‚471580 disk.sda1.in‚0 disk.nvme0c0n1.in‚171.25 disk.drbd1716.in‚0 network.sent‚ 2.713305e+06* disk./.usagePercent‚16.311933752295467 disk.rbd0.in‚0 cpu‚24.076256827689846 disk.dm-1.out‚ 1239.046875 disk.sda.out‚ 1239.046875 disk.drbd2265.in‚0 disk.drbd2265.out‚0$ memory_percent‚85.46867370605469" proc.memory.rssMB‚ 2819.6328125 disk.sda3.in‚0( disk.nvme0c0n1.out‚10994.87255859375 disk.sda2.out‚0 disk.drbd2080.in‚0* proc.memory.percent‚17.209672927856445“Ÿ†’’ ‚(:‡ è÷âÔ„ŽÀ gpu.0.gpu‚37.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚125.548' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.1096 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0êªÃ“ ‚(:’ ÷÷âÔÂàÀ  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.223' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.244600000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0[D ¡Z” ‚(:Ï ÷÷âÔòìÆ* disk./.usagePercent‚15.965516901330682 disk.sda.in‚0 disk.drbd2265.out‚0 disk.sda1.out‚0 disk.rbd1.in‚0 disk.drbd2080.out‚0% disk.nvme0n1.out‚13069.1669921875 disk.sda3.out‚ 1248.7578125 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.dm-1.out‚ 1248.7578125 disk.sda3.in‚0) proc.memory.availableMB‚ 2380.77734375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 175.30078125 disk.drbd2265.in‚0 disk.rbd1.out‚ 6.1171875 disk.dm-0.in‚ 175.30078125" proc.memory.rssMB‚ 2819.6328125 cpu‚24.09387601802519 network.recv‚477445 disk.dm-1.in‚0$ disk./.usageGB‚237.8977394104004" disk.nvme0c0n1.in‚ 175.30078125" disk.dm-0.out‚13069.1669921875 disk.sda2.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.sda1.in‚0' disk.nvme0c0n1.out‚13069.1669921875 disk.sda2.in‚0 disk.drbd1716.out‚0$ memory_percent‚85.46888828277588* proc.memory.percent‚17.209672927856445 network.sent‚ 2.800578e+06 disk.sda.out‚ 1248.7578125 disk.drbd2266.out‚0 proc.cpu.threads‚17}k• ‚(:’ †øâÔñíà  gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚127.824' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.564799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ä4ÝS– ‚(:È †øâÔ¡ûÚ % disk.nvme0n1.out‚17170.0185546875" disk.dm-0.out‚17165.7216796875 disk.drbd1716.out‚0 disk.rbd0.out‚0' proc.memory.availableMB‚ 2381.015625 disk.drbd2266.out‚0 disk.dm-1.in‚0 disk.rbd1.out‚ 6.3447265625! disk.nvme0c0n1.in‚ 186.2109375' disk.nvme0c0n1.out‚17170.0185546875 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda.in‚0 disk.sda3.out‚ 1255.78125 disk.rbd1.in‚0 disk.dm-0.in‚ 186.2109375 network.sent‚ 2.805948e+06* disk./.usagePercent‚16.210817176458907 disk.sda.out‚ 1255.78125 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.drbd2266.in‚0 disk.sda2.out‚0 disk.drbd2080.out‚0$ memory_percent‚85.46743392944336 cpu‚24.111183556605667 proc.cpu.threads‚17 disk.nvme0n1.in‚ 186.2109375 disk.sda3.in‚0 disk.rbd0.in‚0* proc.memory.percent‚17.209672927856445 disk.dm-1.out‚ 1255.78125 disk.sda2.in‚0 network.recv‚481261$ disk./.usageGB‚241.5528907775879" proc.memory.rssMB‚ 2819.6328125 disk.drbd1716.in‚0“ Ps2— - _wandbruntime‚1275  _runtime‚1275LF÷úɘ à ' lr-AdamW/pg2‚0.00011160884201533391 trainer/global_step‚1549 % _timestamp‚1.7883945027162266e+09 _runtime‚1275.696094896 _step‚122 ' lr-AdamW/pg1‚0.00011160884201533391Ÿn'±Í™ Ç % _timestamp‚1.7883945027162266e+09 _runtime‚1275.696094896 _step‚122 ' lr-AdamW/pg1‚0.00011160884201533391 ' lr-AdamW/pg2‚0.00011160884201533391 trainer/global_step‚1549zý 2š - _wandbruntime‚1275  _runtime‚1275þ|î@Ä'› ¾O O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚87291351181312 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚449 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚83772301288393 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85126553457664 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1131715599360 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚82646408673280 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚86258449981952 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20509834 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86111316057088 _runtime‚1275.697417602 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83772124176721 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86111291153408 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21573336 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28468799 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 % _timestamp‚1.7883945027173362e+09 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86258269057024 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28449274 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1180035124224 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚50042137 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86258269057024 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85126553457664 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234311168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1179825716736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28468799 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 % _timestamp‚1.7883945027173362e+09 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86258269057024 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28449274 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1180035124224 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚50042137 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86258269057024 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85126553457664 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234311168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209407488 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1179825716736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚87291351181312 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚449 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚83772301288393 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85126553457664 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1131715599360 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚82646408673280 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚86258449981952 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20509834 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86111316057088 _runtime‚1275.697417602 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83772124176721 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86111291153408 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312{š Ö!2 - _wandbruntime‚1276  _runtime‚1276¬ñsfΞ ÈM TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12989792 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 _runtime‚1276.392453028 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚85179943379968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1132426345984 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 219879936 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1549 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86312369725952 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7532777 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83824748859829 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚86312550664192 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1132590244864 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1132426345984 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚85179960419328 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚87345855597568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚28467266 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚50054098 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚86165317885952 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚82698326396928 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20522569 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚50073651 NADeviceStatsMonitor.on_train_ba’ ž©tch_end/active.small_pool.allocated‚28486817 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21586832 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50073651 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1126439502261 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7533247 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21586832 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚86165292982272 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1180562615296 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20523042 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21586832 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50073651 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1126439502261 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7533247 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21586832 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚86165292982272 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1180562615296 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20523042 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12989792 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 _runtime‚1276.392453028 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚85179943379968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1132426345984 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 219879936 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1549 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86312369725952 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7532777 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83824748859829 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚86312550664192 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1132590244864 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1132426345984 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚85179960419328 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚87345855597568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚28467266 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚50054098 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚86165317885952 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚82698326396928 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20522569|Þ¶+-2  - _wandbruntime‚1276  _runtime‚1276M’*ÛT¡ Î 1loss/FBt2/assignment_loss‚0.20100212830388695 .loss/FRt3/detection_loss‚2.522561504290654 /loss/FRt3/assignment_loss‚4.804395055770874 1loss/SRqqt4/assignment_loss‚2.239187202654606 1loss/SRqqt1/detection_loss‚1.7069411774831158 /loss/FBt1/detection_loss‚3.8100839981010983 epoch‚2 _runtime‚ 1276.39389037 /loss/FRt4/detection_loss‚2.6607330014308292 .loss/FBt3/detection_loss‚3.143325635017111 _step‚125 0loss/FRt2/assignment_loss‚3.5400000479486256 2loss/SRqqt2/assignment_loss‚3.0562997723734657 0loss/FBt4/assignment_loss‚0.3330449916514529 .loss/FRt2/detection_loss‚2.430507930782106 0loss/SRqqt4/detection_loss‚2.411224021121513 .loss/FBt2/detection_loss‚1.077590810256398 % _timestamp‚1.7883945034139283e+09 /loss/FRt1/assignment_loss‚4.192435685396195 1loss/FBt1/assignment_loss‚0.45108799304723235 0loss/SRqqt2/detection_loss‚2.967171248308448 .loss/FBt4/detection_loss‚4.006325395901998 $loss/total_loss‚58.7677179062227 1loss/SRqqt3/assignment_loss‚2.019781104532572 1loss/FBt3/assignment_loss‚0.39519808723707683 /loss/FRt1/detection_loss‚2.4706079506874086 trainer/global_step‚1549 2loss/SRqqt1/assignment_loss‚1.8371872656579529 1loss/SRqqt3/detection_loss‚1.9458647688994042 /loss/FRt4/assignment_loss‚4.545161129368676ÔæiTX¢ Ò /loss/FRt4/detection_loss‚2.6607330014308292 .loss/FBt3/detection_loss‚3.143325635017111 _step‚125 0loss/FRt2/assignment_loss‚3.5400000479486256 2loss/SRqqt2/assignment_loss‚3.0562997723734657 0loss/FBt4/assignment_loss‚0.3330449916514529 .loss/FRt2/detection_loss‚2.430507930782106 0loss/SRqqt4/detection_loss‚2.411224021121513 .loss/FBt2/detection_loss‚1.077590810256398 % _timestamp‚1.7883945034139283e+09 /loss/FRt1/assignment_loss‚4.192435685396195 1loss/FBt1/assignment_loss‚0.45108799304723235 0loss/SRqqt2/detection_loss‚2.967171248308448 .loss/FBt4/detection_loss‚4.006325395901998 $loss/total_loss‚58.7677179062227 1loss/SRqqt3/assignment_loss‚2.019781104532572 1loss/FBt3/assignment_loss‚0.39519808723707683 /loss/FRt1/detection_loss‚2.4706079506874086 trainer/global_step‚1549 2loss/SRqqt1/assignment_loss‚1.8371872656579529 1loss/SRqqt3/detection_loss‚1.9458647688994042 /loss/FRt4/assignment_loss‚4.545161129368676 1loss/FBt2/assignment_loss‚0.20100212830388695 .loss/FRt3/detection_loss‚2.522561504290654 /loss/FRt3/assignment_loss‚4.804395055770874 1loss/SRqqt4/assignment_loss‚2.239187202654606 1loss/SRqqt1/detection_loss‚1.7069411774831158 /loss/FBt1/detection_loss‚3.8100839981010983 epoch‚2 _runtime‚ 1276.39389037}t …’£ ‚(:‡ •øâÔØ·ô  gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚172.377' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4754 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0|É YZ¤ ‚(:Ï •øâÔäñ " disk.dm-0.out‚20002.3076171875 disk.sda2.in‚0 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.dm-1.in‚0 disk.nvme0c0n1.in‚ 274.890625 disk.sda.out‚ 1263.01953125 disk.drbd2265.out‚0 disk.dm-1.out‚ 1263.01953125 disk.rbd1.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0% disk.nvme0n1.out‚20002.3076171875 disk.sda.in‚0 proc.cpu.threads‚17 network.sent‚ 2.811392e+06 disk.drbd2266.out‚0$ memory_percent‚85.46743392944336* proc.memory.percent‚17.209672927856445 disk.drbd2266.in‚0 disk.dm-0.in‚ 274.890625 network.recv‚484807* disk./.usagePercent‚16.311520812142735% disk./.usageGB‚243.05344772338867 disk.sda3.out‚ 1263.01953125 disk.rbd1.out‚ 6.3447265625' proc.memory.availableMB‚ 2381.015625 disk.sda1.out‚0" proc.memory.rssMB‚ 2819.6328125 cpu‚24.127914921157444' disk.nvme0c0n1.out‚20002.3076171875 disk.sda3.in‚0 disk.rbd0.in‚0 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 274.890625-0}’¥ ‚(:‡ ¤øâÔ´®  gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.439' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.4878 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0@? _¦ ‚(:Ô ¤øâÔ쥓 network.sent‚ 2.898983e+06 disk.dm-0.in‚ 338.32421875 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0$ memory_percent‚85.46843528747559 network.recv‚491337' disk.nvme0c0n1.out‚20020.1123046875 disk.sda.out‚ 1269.64453125* proc.memory.percent‚17.209672927856445) disk./.usagePercent‚16.31652831951248" disk.dm-0.out‚20020.1123046875 disk.drbd2080.out‚0% disk.nvme0n1.out‚20020.1123046875 disk.sda1.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 1269.64453125( proc.memory.availableMB‚ 2380.8515625 disk.rbd0.out‚0 disk.sda.in‚0" proc.memory.rssMB‚ 2819.6328125 disk.drbd1716.out‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 1269.64453125 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda1.in‚0 cpu‚24.144781283047948 disk.rbd1.out‚ 6.4072265625$ disk./.usageGB‚243.1280632019043 disk.rbd1.in‚0 disk.drbd2080.in‚0ÿ,Ú2§ - _wandbruntime‚1313  _runtime‚1313ó, ^ɨ à _step‚126 ' lr-AdamW/pg1‚0.00011068426985277508 ' lr-AdamW/pg2‚0.00011068426985277508 trainer/global_step‚1599 % _timestamp‚1.7883945410135071e+09 _runtime‚1313.993312329$<>ƒÍ© Ç % _timestamp‚1.7883945410135071e+09 _runtime‚1313.993312329 _step‚126 ' lr-AdamW/pg1‚0.00011068426985277508 ' lr-AdamW/pg2‚0.00011068426985277508 trainer/global_step‚1599~€ `¤2ª - _wandbruntime‚1313  _runtime‚1313@µø•Ä'« ¾O TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚86403385504129 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚85242294855680 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚88963511256576 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚88963330331648 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88811382596608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 % _timestamp‚1.7883945410145202e+09 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90028385806336 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29352226 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1167444644352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29371751 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1217214714368 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚88963330331648 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29352226 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚85242277816320 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29371751 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚87796066612224 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236408320 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13384686 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90028622214656 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚51600362 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 trainer/global_step‚1599 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86403562615801 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1167444644352 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22248138 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚88963511256576 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚87796066612224 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21148545 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚88811407500288 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7764324 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21149010 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22248136 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13384683 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22248136 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22248138 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1161107687809 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _runtime‚1313.998215381 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1161267760121 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1167280758784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51619889 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1217003209728 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚87796049572864 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1167280758784 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86403562615801 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1167444644352 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22248138 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚88963511256576 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚87796066612224 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21148545 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚88811407500288 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7764324 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21149010 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22248136 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13384683 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22248136 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22248138 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1161107687809 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _runtime‚1313.998215381 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1161267760121 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1167280758784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51619889 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1217003209728 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚87796049572864 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1167280758784 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚88963330331648 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88811382596608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 % _timestamp‚1.7883945410145202e+09 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90028385806336 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29352226 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1167444644352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29371751 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1217214714368 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚88963330331648 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29352226 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚85242277816320 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979Kù*Á2­ - _wandbruntime‚1314  _runtime‚1314ÿn”iÎ&® ÈM NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22261634 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚51651403 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90083132678656 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29370218 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21161215 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚88865409329152 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 % _timestamp‚1.7883945417003927e+09 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1167991505408 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1217748253184 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22261634 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29370218 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚85294212579328 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚87849439495168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29389769 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚87849439495168 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚85294195539968 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 _runtime‚1314.680502818 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7768473 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚86456187299117 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚87849456534528 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚87849456534528 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90083364879360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1217955550208 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13392742 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7768938 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21161683 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1167991505408 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚88865384425472 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1161814647269 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚51631850 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚86456010187237 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13392745 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89017611938816 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51651403 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51631850 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1599 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89017611938816 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 _step‚128 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1168155404288 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚468 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89017611938816 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51651403 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51631850 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1599 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89017611938816 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 _step‚128 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1168155404288 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚468 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚51651403 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 232200704 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90083132678656 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29370218 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21161215 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚88865409329152 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207297024 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 % _timestamp‚1.7883945417003927e+09 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1167991505408 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1217748253184 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22261634 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29370218 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚85294212579328 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚87849439495168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29389769 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚87849439495168 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚85294195539968 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 _runtime‚1314.680502818 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7768473 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚86456187299117 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚87849456534528 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚87849456534528 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90083364879360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1217955550208 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚13392742 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7768938 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21161683 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1167991505408 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚88865384425472 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1161814647269 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚51631850 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚86456010187237 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13392745 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360€´ÑÙ'2° - _wandbruntime‚1314  _runtime‚1314`¶Ñ€P± Ê trainer/global_step‚1599 /loss/FRt4/assignment_loss‚4.263252596701345 2loss/SRqqt1/assignment_loss‚1.5186919008122115 2loss/SRqqt3/assignment_loss‚2.0936016602317493 /loss/FRt4/detection_loss‚2.4980356263858017 /loss/FRt2/detection_loss‚2.5060267412662505 /loss/SRqqt1/detection_loss‚1.85233288878178 0loss/SRqqt3/detection_loss‚1.742942064702511 _step‚129 0loss/FRt2/assignment_loss‚3.3412932682037355 1loss/SRqqt2/assignment_loss‚2.792479469662621 0loss/FBt2/assignment_loss‚0.2374572529369651 .loss/FBt3/detection_loss‚3.810912847518921 epoch‚2 0loss/FRt1/assignment_loss‚3.6764572401131903 1loss/SRqqt4/assignment_loss‚1.983736345524429 0loss/SRqqt4/detection_loss‚2.319477858722851 % _timestamp‚1.7883945417022984e+09 0loss/FBt4/assignment_loss‚0.7332911573768277 /loss/FRt3/assignment_loss‚4.298508092073294 .loss/FRt3/detection_loss‚2.568151836211865 %loss/total_loss‚60.37349245331753 .loss/FRt1/detection_loss‚2.204691937858505 0loss/SRqqt2/detection_loss‚2.959312534048444 /loss/FBt2/detection_loss‚1.2236072849795343 _runtime‚1314.681989633 0loss/FBt3/assignment_loss‚0.4924564677138206 .loss/FBt4/detection_loss‚6.681734681129456 /loss/FBt1/assignment_loss‚0.358233565017872 .loss/FBt1/detection_loss‚4.216807135343552omÔU² Ï 1loss/SRqqt2/assignment_loss‚2.792479469662621 0loss/FBt2/assignment_loss‚0.2374572529369651 .loss/FBt3/detection_loss‚3.810912847518921 epoch‚2 0loss/FRt1/assignment_loss‚3.6764572401131903 1loss/SRqqt4/assignment_loss‚1.983736345524429 0loss/SRqqt4/detection_loss‚2.319477858722851 % _timestamp‚1.7883945417022984e+09 0loss/FBt4/assignment_loss‚0.7332911573768277 /loss/FRt3/assignment_loss‚4.298508092073294 .loss/FRt3/detection_loss‚2.568151836211865 %loss/total_loss‚60.37349245331753 .loss/FRt1/detection_loss‚2.204691937858505 0loss/SRqqt2/detection_loss‚2.959312534048444 /loss/FBt2/detection_loss‚1.2236072849795343 _runtime‚1314.681989633 0loss/FBt3/assignment_loss‚0.4924564677138206 .loss/FBt4/detection_loss‚6.681734681129456 /loss/FBt1/assignment_loss‚0.358233565017872 .loss/FBt1/detection_loss‚4.216807135343552 trainer/global_step‚1599 /loss/FRt4/assignment_loss‚4.263252596701345 2loss/SRqqt1/assignment_loss‚1.5186919008122115 2loss/SRqqt3/assignment_loss‚2.0936016602317493 /loss/FRt4/detection_loss‚2.4980356263858017 /loss/FRt2/detection_loss‚2.5060267412662505 /loss/SRqqt1/detection_loss‚1.85233288878178 0loss/SRqqt3/detection_loss‚1.742942064702511 _step‚129 0loss/FRt2/assignment_loss‚3.3412932682037355ñ‰PÃ’³ ‚(:‡ ³øâÔöÓ»  gpu.0.gpu‚56.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.663' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.9326 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚09ãä ´ ‚(:wîêZUÓ ³øâÔ‡´• disk.sda2.in‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.rbd1.out‚ 6.4072265625 disk.dm-1.in‚0 disk.drbd2080.in‚0% disk.nvme0n1.out‚20200.3505859375 disk.sda.out‚ 1287.07421875 disk.drbd2265.out‚0 disk.dm-0.in‚ 338.32421875 disk.rbd0.out‚0 disk.sda3.in‚0 proc.cpu.threads‚17 network.sent‚ 2.905122e+06 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.dm-1.out‚ 1287.07421875 disk.rbd1.in‚0 cpu‚24.160826338504414 disk.drbd1716.in‚0 disk.sda1.out‚0 network.recv‚495648) disk./.usagePercent‚16.31452045242143" disk./.usageGB‚243.09814453125 disk.rbd0.in‚0 disk.sda3.out‚ 1287.07421875) proc.memory.availableMB‚ 2380.62109375" disk.nvme0c0n1.in‚ 338.32421875' disk.nvme0c0n1.out‚20200.3505859375 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.drbd2266.out‚0 disk.sda1.in‚0* proc.memory.percent‚17.209672927856445" disk.dm-0.out‚20200.3505859375$ memory_percent‚85.46984195709229" proc.memory.rssMB‚ 2819.6328125ñ€d’µ ‚(:‡ ÂøâÔõãÿ  gpu.0.gpu‚40.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚124.046' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.8092 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Þj–†U¶ ‚(:Ê ÂøâÔ¸´ç  network.sent‚ 2.99174e+06) disk./.usagePercent‚16.31193093621259" disk.nvme0c0n1.in‚ 338.32421875% disk.nvme0c0n1.out‚20272.11328125# disk.nvme0n1.out‚20272.11328125 disk.dm-1.in‚0* proc.memory.percent‚17.209672927856445 proc.cpu.threads‚17 disk.dm-1.out‚ 1300.2109375 disk.rbd1.in‚0 disk.rbd1.out‚ 6.611328125 disk.rbd0.in‚0 disk.sda.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 1300.2109375 network.recv‚501654 disk.dm-0.in‚ 338.32421875 disk.drbd1716.out‚0 disk.sda.out‚ 1300.2109375 disk.sda1.out‚0 cpu‚24.176590567944153 disk.dm-0.out‚20272.11328125 disk.nvme0n1.in‚ 338.32421875 disk.sda2.out‚0 disk.rbd0.out‚0 disk.sda2.in‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0$ memory_percent‚85.46721935272217) proc.memory.availableMB‚ 2381.05078125$ disk./.usageGB‚243.0595588684082 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.sda3.in‚0" proc.memory.rssMB‚ 2819.6328125‚S¡—’· ‚(:‡ ÑøâÔôÍØ  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚167.828' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5656 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÐÝUtQ¸ ‚(:Æ ÑøâÔê¹ì  disk.rbd1.in‚0 disk.dm-0.out‚ 20272.265625 proc.cpu.threads‚17 disk.dm-1.out‚ 1314.37890625 disk.rbd1.out‚ 6.697265625" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875* proc.memory.percent‚17.209672927856445 cpu‚24.19198915193076) disk./.usagePercent‚16.31193093621259 disk.drbd2080.in‚0 network.sent‚ 2.996996e+06! disk.nvme0n1.out‚ 20272.265625 disk.sda1.in‚0 disk.dm-1.in‚0 disk.sda3.in‚0 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚85.46719551086426 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda.in‚0 disk.drbd2265.out‚0# disk.nvme0c0n1.out‚ 20272.265625" proc.memory.rssMB‚ 2819.6328125 disk.sda.out‚ 1314.37890625 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd1716.in‚0 disk.drbd2265.in‚0( proc.memory.availableMB‚ 2381.0546875 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.sda3.out‚ 1314.37890625 network.recv‚504687$ disk./.usageGB‚243.0595588684082 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 338.32421875æj³2¹ - _wandbruntime‚1352  _runtime‚1352öŸwɺ à _runtime‚1352.751395843 _step‚130 ' lr-AdamW/pg1‚0.00010971753137076432 ' lr-AdamW/pg2‚0.00010971753137076432 trainer/global_step‚1649 % _timestamp‚1.7883945797716043e+09}@λ È ' lr-AdamW/pg1‚0.00010971753137076432 ' lr-AdamW/pg2‚0.00010971753137076432 trainer/global_step‚1649 % _timestamp‚1.7883945797716043e+09 _runtime‚1352.751395843 _step‚130‚zóÍ2¼ - _wandbruntime‚1352  _runtime‚1352ˆÀèÄ'½ ¾O ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1202845918208 % _timestamp‚1.7883945797726657e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92765914116096 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92765671416320 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22922938 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91511498943488 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30255178 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796096 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 trainer/global_step‚1649 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 _runtime‚1352.752719922 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90465545688064 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1202845918208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21785955 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91511474039808 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1196642905129 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1196482832817 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚91668391606272 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21785458 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13787636 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22922936 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53197641 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1254415172608 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚87838163998720 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 _step‚131 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22922938 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚91668572531200 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90465562727424 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚53178114 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚91668572531200 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30255178 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13787633 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚494 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1203009803776 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚91668391606272 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 242699776 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623í’Í7É'¾ ÃO trainer/global_step‚1649 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 _runtime‚1352.752719922 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90465545688064 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1202845918208 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21785955 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91511474039808 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1196642905129 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1196482832817 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚91668391606272 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21785458 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13787636 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22922936 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53197641 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1254415172608 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚87838163998720 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 _step‚131 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22922938 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚91668572531200 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90465562727424 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚53178114 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚91668572531200 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30255178 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13787633 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚494 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1203009803776 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚91668391606272 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 242699776 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1202845918208 % _timestamp‚1.7883945797726657e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92765914116096 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92765671416320 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22922938 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91511498943488 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30255178 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217796096 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464ƒÞá•2¿ - _wandbruntime‚1353  _runtime‚1353q4)¬ÌÀ ÈM FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1254934510592 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚90518952649728 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚90518935610368 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚53209602 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1197349864797 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚495 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 trainer/global_step‚1649 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1197189792277 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1203556664832 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚89087448626525 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7883945804614062e+09 _step‚132 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚87890081722368 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 _runtime‚1353.441509536 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚498 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚91722673213440 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚90518935610368 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1255154390528 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1203720563712 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22936434 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚30292721 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30273170 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53229155 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21798654 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚89087271514645 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚91565500772352 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8002959 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1203720563712 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13795695 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚91722673213440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚90518952649728 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22936432 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21798156 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92820655162880 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553ÁÓ“Ó&Á ÍM UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1203556664832 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚89087448626525 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7883945804614062e+09 _step‚132 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚87890081722368 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 _runtime‚1353.441509536 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚498 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659456 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚91722673213440 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚90518935610368 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1255154390528 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1203720563712 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22936434 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783616 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚30292721 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30273170 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53229155 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21798654 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚89087271514645 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚91565500772352 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8002959 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1203720563712 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13795695 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚91722673213440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚90518952649728 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22936432 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21798156 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92820655162880 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1254934510592 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚90518952649728 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚90518935610368 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚53209602 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1197349864797 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚495 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 trainer/global_step‚1649 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1197189792277 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264„­bp–2 - _wandbruntime‚1353  _runtime‚1353¶ù»9Rà Ì 2loss/SRqqt3/assignment_loss‚1.8724194567243593 .loss/FRt1/detection_loss‚2.260967253175165 trainer/global_step‚1649 /loss/FRt3/assignment_loss‚4.192234720244552 0loss/FBt3/assignment_loss‚0.4815529640940781 /loss/FRt3/detection_loss‚2.4344867716232934 1loss/SRqqt1/detection_loss‚1.7060407765485621 .loss/FBt1/detection_loss‚3.251905204852422 # _timestamp‚1.78839458046323e+09 /loss/FRt2/assignment_loss‚3.660597672433029 0loss/FBt1/assignment_loss‚0.5138795826061444 /loss/FRt2/detection_loss‚2.5206539805656596 .loss/FBt4/detection_loss‚4.669069903237479 0loss/FRt1/assignment_loss‚3.8782071002892087 2loss/SRqqt1/assignment_loss‚1.8753315626471132 .loss/FRt4/detection_loss‚2.381405228435403 0loss/SRqqt3/detection_loss‚2.004987899558384 %loss/total_loss‚57.42894486641594 1loss/SRqqt4/assignment_loss‚2.384571952988272 1loss/SRqqt4/detection_loss‚2.1666809939171956 /loss/FBt2/detection_loss‚1.2323754055655374 .loss/FBt3/detection_loss‚2.502168656210415 _step‚133 /loss/FRt4/assignment_loss‚4.443690183139084 0loss/FBt2/assignment_loss‚0.2801602108718738 0loss/FBt4/assignment_loss‚0.5847975103388308 _runtime‚1353.442862098 1loss/SRqqt2/assignment_loss‚2.748484282658018 1loss/SRqqt2/detection_loss‚3.3822755936918587 epoch‚2Ä·BWÄ Ñ 2loss/SRqqt3/assignment_loss‚1.8724194567243593 .loss/FRt1/detection_loss‚2.260967253175165 trainer/global_step‚1649 /loss/FRt3/assignment_loss‚4.192234720244552 0loss/FBt3/assignment_loss‚0.4815529640940781 /loss/FRt3/detection_loss‚2.4344867716232934 1loss/SRqqt1/detection_loss‚1.7060407765485621 .loss/FBt1/detection_loss‚3.251905204852422 # _timestamp‚1.78839458046323e+09 /loss/FRt2/assignment_loss‚3.660597672433029 0loss/FBt1/assignment_loss‚0.5138795826061444 /loss/FRt2/detection_loss‚2.5206539805656596 .loss/FBt4/detection_loss‚4.669069903237479 0loss/FRt1/assignment_loss‚3.8782071002892087 2loss/SRqqt1/assignment_loss‚1.8753315626471132 .loss/FRt4/detection_loss‚2.381405228435403 0loss/SRqqt3/detection_loss‚2.004987899558384 %loss/total_loss‚57.42894486641594 1loss/SRqqt4/assignment_loss‚2.384571952988272 1loss/SRqqt4/detection_loss‚2.1666809939171956 /loss/FBt2/detection_loss‚1.2323754055655374 .loss/FBt3/detection_loss‚2.502168656210415 _step‚133 /loss/FRt4/assignment_loss‚4.443690183139084 0loss/FBt2/assignment_loss‚0.2801602108718738 0loss/FBt4/assignment_loss‚0.5847975103388308 _runtime‚1353.442862098 1loss/SRqqt2/assignment_loss‚2.748484282658018 1loss/SRqqt2/detection_loss‚3.3822755936918587 epoch‚2…Bìˆ]’Å ‚(:‡ àøâÔ×™¨  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚125.106' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.0212 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚00€WË_Æ ‚(:Ô àøâÔ„»Â# disk.dm-0.out‚20276.53662109375 disk.sda2.in‚0 disk.nvme0n1.in‚ 338.32421875 disk.rbd1.out‚ 6.759765625" proc.memory.rssMB‚ 2819.6328125 disk.dm-1.in‚0 disk.drbd1716.out‚0& disk.nvme0n1.out‚20276.53662109375 disk.sda.in‚0 disk.sda.out‚ 1328.9296875 disk.drbd2265.out‚0 cpu‚24.207357680165746 proc.cpu.threads‚17 disk.sda2.out‚0 network.sent‚ 3.016678e+06 network.recv‚509849* disk./.usagePercent‚16.311924536024232% disk./.usageGB‚243.05946350097656 disk.dm-0.in‚ 338.32421875' proc.memory.availableMB‚ 2380.296875 disk.sda1.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 1328.9296875 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.sda1.out‚0 disk.rbd1.in‚0( disk.nvme0c0n1.out‚20276.53662109375 disk.drbd2266.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.dm-1.out‚ 1328.9296875" disk.nvme0c0n1.in‚ 338.32421875$ memory_percent‚85.47182083129883* proc.memory.percent‚17.209672927856445€_ôEÇ ‚(:„ ïøâÔëôí  gpu.0.gpu‚5.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚123.51' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.702 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Mº/\È ‚(:Ñ ïøâÔƒµ‚ disk.rbd0.in‚0 disk.dm-0.in‚ 338.32421875 disk.rbd0.out‚0 disk.sda.in‚0 disk.drbd2265.out‚0 disk.sda3.in‚0( disk.nvme0c0n1.out‚20276.75927734375 disk.nvme0n1.in‚ 338.32421875 disk.drbd2266.out‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1344.890625 cpu‚24.22199957315781 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.sda2.out‚0 disk.sda.out‚ 1344.890625 disk.dm-1.in‚0# memory_percent‚85.4712963104248( proc.memory.availableMB‚ 2380.3828125# proc.memory.rssMB‚ 2819.63671875 disk.sda3.out‚ 1344.890625 disk.rbd1.out‚ 6.759765625 disk.drbd1716.in‚0% disk./.usageGB‚243.05942916870117 disk.rbd1.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2265.in‚0* proc.memory.percent‚17.209696769714355 network.sent‚ 3.090906e+06 network.recv‚516156* disk./.usagePercent‚16.311922231956423 disk.sda2.in‚0& disk.nvme0n1.out‚20276.75927734375 proc.cpu.threads‚17" disk.nvme0c0n1.in‚ 338.32421875# disk.dm-0.out‚20276.75927734375¦-pÂ2É - _wandbruntime‚1391  _runtime‚1391³èx©ÉÊ Ã ' lr-AdamW/pg1‚0.00010870943372932629 ' lr-AdamW/pg2‚0.00010870943372932629 trainer/global_step‚1699 % _timestamp‚1.7883946182593846e+09 _runtime‚1391.240202819 _step‚134`e¹½ÎË È _step‚134 ' lr-AdamW/pg1‚0.00010870943372932629 ' lr-AdamW/pg2‚0.00010870943372932629 trainer/global_step‚1699 % _timestamp‚1.7883946182593846e+09 _runtime‚1391.240202819†:´Û2Ì - _wandbruntime‚1391  _runtime‚1391/@úÄ'Í ¾O XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚93135041803264 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8234917 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚95503163989504 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31158130 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31177655 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31158130 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚94373633805824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22425503 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚475 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚94211565483008 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14190583 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 _runtime‚1391.240293821 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22425028 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23597738 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23597736 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚93135058842624 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚472 trainer/global_step‚1699 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚94373633805824 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23597738 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1232018050137 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1291360001024 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚90434067220480 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚94373452880896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1238574963200 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1238574963200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31177655 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14190586 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚94373452880896 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1291573602816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0\×ûšLÎ ÃO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1238574963200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31177655 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14190586 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚94373452880896 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1291573602816 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚93135041803264 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8234917 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚95503163989504 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31158130 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31177655 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train&øuÔ}_batch_start/active.large_pool.peak‚2513 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31158130 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚94373633805824 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22425503 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚475 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚94211565483008 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14190583 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 _runtime‚1391.240293821 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22425028 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23597738 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23597736 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚93135058842624 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚472 trainer/global_step‚1699 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚94373633805824 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23597738 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1232018050137 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1291360001024 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚90434067220480 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚94373452880896 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1238574963200‡NÃBW2Ï - _wandbruntime‚1391  _runtime‚1391D ˜úÍ&Ð ÇM REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚23611234 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1292311882752 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1232564937285 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 _runtime‚1391.915357727 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚94427734488064 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14198642 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31176122 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94265567311872 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1292106682880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚94427734488064 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23611232 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚93188448764928 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚447 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1239285723136 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163898880 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22438188 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚450 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 205199872 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1232725009805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚93188431725568 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31176122 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1239121824256 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8239543 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚90485984944128 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31195673 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚90485967904768 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22437738 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23611234 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚54787354 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1239285723136 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 _step‚136 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚91718532842053 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54806907 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230103552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54806907 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚94427553549824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚95557904098304 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94265592215552 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31195673 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 205199872 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1232725009805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚93188431725568 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31176122 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1239121824256 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8239543 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚90485984944128 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324224 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31195673 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚90485967904768 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22437738 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23611234 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚54787354 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1239285723136 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 _step‚136 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚91718532842053 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54806907 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 230103552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54806907 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚94427553549824 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚95557904098304 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94265592215552 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31195673 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14198642 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31176122 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94265567311872 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1292106682880 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚94427734488064 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23611232 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚93188448764928 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚447 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0ˆ±ë±±2Ò - _wandbruntime‚1391  _runtime‚1391G‹ WÓ Ñ 0loss/FBt3/assignment_loss‚0.3794924492769951 0loss/SRqqt1/detection_loss‚1.698126263861303 .loss/FBt1/detection_loss‚5.187452040115992 trainer/global_step‚1699 2loss/SRqqt2/assignment_loss‚2.7845371634352443 %loss/total_loss‚59.65442776823505 /loss/FRt3/assignment_loss‚4.365414515718237 1loss/SRqqt2/detection_loss‚2.6577431448534425 1loss/SRqqt4/detection_loss‚2.1353236449801405 epoch‚2 /loss/FRt1/assignment_loss‚3.482062599311272 0loss/FBt4/assignment_loss‚0.7101677777837418 -loss/FRt2/detection_loss‚2.58894101512574 2loss/SRqqt1/assignment_loss‚1.6733409945611601 2loss/SRqqt4/assignment_loss‚2.1344010514085707 /loss/FBt2/detection_loss‚1.0613347445344299 _runtime‚1391.916515452 /loss/FRt3/detection_loss‚2.3459473480651907 .loss/FRt4/detection_loss‚2.518970725300548 1loss/SRqqt3/detection_loss‚1.9902597837920846 % _timestamp‚1.7883946189368553e+09 0loss/FRt2/assignment_loss‚3.9339417041767213 0loss/FRt4/assignment_loss‚3.8438422869849993 _step‚137 2loss/SRqqt3/assignment_loss‚2.0944127490551305 0loss/FBt1/assignment_loss‚0.3661682873343428 .loss/FRt1/detection_loss‚2.401428046015402 /loss/FBt3/detection_loss‚3.2130182050168514 .loss/FBt4/detection_loss‚5.743360545237859 1loss/FBt2/assignment_loss‚0.34474068228965654@ôà·\Ô Ö /loss/FRt3/assignment_loss‚4.365414515718237 1loss/SRqqt2/detection_loss‚2.6577431448534425 1loss/SRqqt4/detection_loss‚2.1353236449801405 epoch‚2 /loss/FRt1/assignment_loss‚3.482062599311272 0loss/FBt4/assignment_loss‚0.7101677777837418 -loss/FRt2/detection_loss‚2.58894101512574 2loss/SRqqt1/assignment_loss‚1.6733409945611601 2loss/SRqqt4/assignment_loss‚2.1344010514085707 /loss/FBt2/detection_loss‚1.0613347445344299 _runtime‚1391.916515452 /loss/FRt3/detection_loss‚2.3459473480651907 .loss/FRt4/detection_loss‚2.518970725300548 1loss/SRqqt3/detection_loss‚1.9902597837920846 % _timestamp‚1.7883946189368553e+09 0loss/FRt2/assignment_loss‚3.9339417041767213 0loss/FRt4/assignment_loss‚3.8438422869849993 _step‚137 2loss/SRqqt3/assignment_loss‚2.0944127490551305 0loss/FBt1/assignment_loss‚0.3661682873343428 .loss/FRt1/detection_loss‚2.401428046015402 /loss/FBt3/detection_loss‚3.2130182050168514 .loss/FBt4/detection_loss‚5.743360545237859 1loss/FBt2/assignment_loss‚0.34474068228965654 0loss/FBt3/assignment_loss‚0.3794924492769951 0loss/SRqqt1/detection_loss‚1.698126263861303 .loss/FBt1/detection_loss‚5.187452040115992 trainer/global_step‚1699 2loss/SRqqt2/assignment_loss‚2.7845371634352443 %loss/total_loss‚59.65442776823505‰Üñ½tœÕ ‚(:‘ þøâÔçȃ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚178.569' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚35.71379999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‡²öÐZÖ ‚(:Ï þøâÔÍ£Ò disk.dm-1.out‚ 1361.953125" disk.dm-0.out‚20284.0927734375$ memory_percent‚85.47282218933105# proc.memory.rssMB‚ 2819.63671875* proc.memory.percent‚17.209696769714355 proc.cpu.threads‚17 disk.drbd2265.in‚0 cpu‚24.23634975835718% disk.nvme0n1.out‚20284.0927734375 disk.sda.out‚ 1361.953125 disk.sda3.in‚0 disk.rbd1.in‚0( proc.memory.availableMB‚ 2380.1328125 disk.dm-0.in‚ 338.32421875 disk.rbd0.in‚0 disk.dm-1.in‚0 disk.sda2.out‚0 network.recv‚520176 disk.rbd0.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda3.out‚ 1361.953125" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd1716.in‚0 network.sent‚ 3.096296e+06 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.sda1.out‚0% disk./.usageGB‚243.06221389770508 disk.sda.in‚0 disk.drbd2266.out‚0) disk./.usagePercent‚16.31210911745642 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.rbd1.out‚ 6.9013671875' disk.nvme0c0n1.out‚20284.0927734375 disk.sda2.in‚0 disk.drbd1716.out‚0ß«g‘’× ‚(:‡ ùâÔ…“é  gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚171.023' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.2046 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ùF^Ø ‚(:Ó ùâÔþ„% disk./.usageGB‚243.06219482421875) disk./.usagePercent‚16.31210783741875 disk.sda3.out‚ 1379.8515625 disk.rbd1.in‚0 disk.sda2.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.drbd2265.in‚0' disk.nvme0c0n1.out‚20284.6689453125 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.dm-1.in‚0$ memory_percent‚85.47158241271973" disk.nvme0c0n1.in‚ 338.32421875 disk.rbd0.in‚0 disk.dm-0.in‚ 338.32421875 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1379.8515625* proc.memory.percent‚17.209696769714355 network.recv‚524523" disk.dm-0.out‚20284.6689453125 disk.drbd2080.in‚0% disk.nvme0n1.out‚20284.6689453125 disk.sda.out‚ 1379.8515625# proc.memory.rssMB‚ 2819.63671875 cpu‚24.250881223686306 disk.drbd1716.out‚0 disk.drbd2266.out‚0( proc.memory.availableMB‚ 2380.3359375 proc.cpu.threads‚17 network.sent‚ 3.181695e+06 disk.sda3.in‚0 disk.rbd1.out‚ 6.9013671875£zîI’Ù ‚(:‡ œùâÔºßê  gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚125.932' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.1864 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0HY úÚ ‚(:È œùâÔ×õÆ  disk.rbd1.out‚ 7.0498046875 proc.cpu.threads‚17 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.sda3.out‚1396.375" disk.nvme0c0n1.in‚ 338.32421875* proc.memory.percent‚17.209696769714355 network.r Ðt*Yecv‚529232% disk./.usageGB‚243.06184005737305 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.sda.out‚1396.375 disk.drbd2265.in‚0' disk.nvme0c0n1.out‚20285.9189453125) proc.memory.availableMB‚ 2380.08203125 disk.drbd2080.out‚0% disk.nvme0n1.out‚20285.9189453125# proc.memory.rssMB‚ 2819.63671875 cpu‚24.265033444732154 disk.dm-0.in‚ 338.32421875" disk.dm-0.out‚20285.9189453125 disk.sda1.in‚0 disk.sda3.in‚0 disk.rbd1.in‚0* disk./.usagePercent‚16.312084028718065 disk.sda.in‚0 disk.drbd2266.out‚0 network.sent‚ 3.18818e+06 disk.dm-1.in‚0 disk.dm-1.out‚1396.375 disk.sda2.in‚0 disk.rbd0.out‚0 disk.drbd2265.out‚0$ memory_percent‚85.47313213348389 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda1.out‚0§€Ñ2Û - _wandbruntime‚1429  _runtime‚1429uvNÉÜ Ã _runtime‚1429.351222045 _step‚138 ' lr-AdamW/pg1‚0.00010766081862053418 ' lr-AdamW/pg2‚0.00010766081862053418 trainer/global_step‚1749 % _timestamp‚1.7883946563713472e+09FÔQuÎÝ È ' lr-AdamW/pg1‚0.00010766081862053418 ' lr-AdamW/pg2‚0.00010766081862053418 trainer/global_step‚1749 % _timestamp‚1.7883946563713472e+09 _runtime‚1429.351222045 _step‚138Š;+2Þ - _wandbruntime‚1429  _runtime‚1429âK’³Â'ß ¼O OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1328571236864 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚32080607 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23064416 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97078514155520 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8471382 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1267393195145 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1267233122833 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14593536 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚98240475057152 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 # _timestamp‚1.78839465637251e+09 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1274140122624 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97078514155520 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚98240228163072 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23064918 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚56333618 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚95804537918464 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚499 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93029953402880 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32080607 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32061082 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97078695080448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚93029936363520 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚94297346598025 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚97078695080448 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1273976237056 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32061082 _runtime‚1429.355273582 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24272536 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24272538 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24272536 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1273976237056 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚56353145 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚96911656926208 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚95804554957824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1274140122624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚95804554957824 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 246894080 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1328793227264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221990400 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14593533 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚95804537918464 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1274140122624 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97078514155520 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚98240228163072 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23764 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072312 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23064918 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚56333618 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚95804537918464 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚499 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93029953402880 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32080607 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32061082 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97078695080448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚93029936363520 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚94297346598025 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚97078695080448 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1273976237056 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32061082 _runtime‚1429.355273582 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24272536 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24272538 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659456 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324224 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24272536 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1273976237056 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚56353145 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚96911656926208 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚95804554957824 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1274140122624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚95804554957824 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 246894080 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1328793227264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221990400 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23764 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14593533 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚95804537918464 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97078514155520 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8471382 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1267393195145 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1267233122833 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14593536 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚98240475057152 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0‹—=¶À2á - _wandbruntime‚1430  _runtime‚1430Ô¸ìÎ&â ÈM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚95857944880128 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚98295209812480 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚95857927840768 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14601595 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14601592 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1329306273792 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚94349794169461 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚97132614824448 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 % _timestamp‚1.7883946570562563e+09 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97132795762688 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1274686983680 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98294965028864 _runtime‚1430.036494045 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23077166 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24286032 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚93081854087168 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24286034 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1274850882560 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1274686983680 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24286034 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1268100154813 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32079074 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1274850882560 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8475574 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1329526153728 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 _step‚140 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23077640 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚474 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97132795762688 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚471 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1749 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24286032 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚95857927840768 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32079074 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚96965658755072 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚56384659 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56384659 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚96965683658752 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚93081871126528 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24286034 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1274850882560 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1274686983680 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24286034 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 \JDeviceStatsMonitor.on_traiàž;ûö"n_batch_end/requested_bytes.small_pool.allocated‚ 1268100154813 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32079074 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1274850882560 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8475574 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1329526153728 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 _step‚140 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23077640 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚474 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97132795762688 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚471 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1749 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24286032 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚95857927840768 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32079074 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚96965658755072 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚56384659 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56384659 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚96965683658752 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚93081871126528 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚95857944880128 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚98295209812480 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚95857927840768 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14601595 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783616 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14601592 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1329306273792 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚94349794169461 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚97132614824448 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 % _timestamp‚1.7883946570562563e+09 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97132795762688 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1274686983680 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98294965028864 _runtime‚1430.036494045 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23077166 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24286032 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464Œ ¤2ä - _wandbruntime‚1430  _runtime‚1430Ѓ• Så Í 1loss/SRqqt4/assignment_loss‚2.340207149088383 0loss/FBt2/assignment_loss‚0.2280731072432083 /loss/FRt2/detection_loss‚2.3772196712184557 0loss/SRqqt1/detection_loss‚1.880982319073331 %loss/total_loss‚57.50230820424605 0loss/FRt2/assignment_loss‚3.1354024153721483 .loss/FRt3/assignment_loss‚4.08003414093062 2loss/SRqqt3/assignment_loss‚1.9211798340759494 /loss/FRt1/detection_loss‚2.2883451174134795 .loss/FBt3/detection_loss‚2.326204615575225 % _timestamp‚1.7883946570580332e+09 _step‚141 .loss/FBt1/detection_loss‚4.398355573415756 .loss/FRt4/detection_loss‚2.647503288998001 1loss/SRqqt3/detection_loss‚1.8801839672537013 0loss/SRqqt4/detection_loss‚2.389526514336467 trainer/global_step‚1749 _runtime‚1430.037659885 2loss/SRqqt1/assignment_loss‚1.8456600433996608 1loss/FBt4/assignment_loss‚0.47585020595256033 .loss/FBt4/detection_loss‚4.680414577325185 epoch‚2 0loss/FBt1/assignment_loss‚0.6039009572012017 0loss/FBt3/assignment_loss‚0.6093478337622946 .loss/FRt3/detection_loss‚2.416752488114113 /loss/FRt4/assignment_loss‚4.671815694063559 0loss/SRqqt2/detection_loss‚2.770663247867064 /loss/FBt2/detection_loss‚0.8247861342187405 /loss/FRt1/assignment_loss‚3.912891235558883 2loss/SRqqt2/assignment_loss‚2.7970080727880653Ñz[TXæ Ò /loss/FRt1/assignment_loss‚3.912891235558883 2loss/SRqqt2/assignment_loss‚2.7970080727880653 1loss/SRqqt4/assignment_loss‚2.340207149088383 0loss/FBt2/assignment_loss‚0.2280731072432083 /loss/FRt2/detection_loss‚2.3772196712184557 0loss/SRqqt1/detection_loss‚1.880982319073331 %loss/total_loss‚57.50230820424605 0loss/FRt2/assignment_loss‚3.1354024153721483 .loss/FRt3/assignment_loss‚4.08003414093062 2loss/SRqqt3/assignment_loss‚1.9211798340759494 /loss/FRt1/detection_loss‚2.2883451174134795 .loss/FBt3/detection_loss‚2.326204615575225 % _timestamp‚1.7883946570580332e+09 _step‚141 .loss/FBt1/detection_loss‚4.398355573415756 .loss/FRt4/detection_loss‚2.647503288998001 1loss/SRqqt3/detection_loss‚1.8801839672537013 0loss/SRqqt4/detection_loss‚2.389526514336467 trainer/global_step‚1749 _runtime‚1430.037659885 2loss/SRqqt1/assignment_loss‚1.8456600433996608 1loss/FBt4/assignment_loss‚0.47585020595256033 .loss/FBt4/detection_loss‚4.680414577325185 epoch‚2 0loss/FBt1/assignment_loss‚0.6039009572012017 0loss/FBt3/assignment_loss‚0.6093478337622946 .loss/FRt3/detection_loss‚2.416752488114113 /loss/FRt4/assignment_loss‚4.671815694063559 0loss/SRqqt2/detection_loss‚2.770663247867064 /loss/FBt2/detection_loss‚0.8247861342187405Æ“?øç ‚(:’ «ùâÔ¾…ì  gpu.0.gpu‚39.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.361' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.072199999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ïŠUaè ‚(:Ö «ùâÔ¯÷Õ disk.sda.in‚0 disk.drbd2266.out‚0" disk.dm-0.out‚20286.1689453125 disk.sda2.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.drbd2266.in‚0 disk.drbd2265.in‚0( proc.memory.availableMB‚ 2379.7734375 proc.cpu.threads‚17 disk.drbd2080.out‚0 disk.sda.out‚ 1408.89453125 disk.dm-1.in‚0 disk.dm-1.out‚ 1408.89453125" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875% disk./.usageGB‚243.06194305419922 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.sda1.out‚0* proc.memory.percent‚17.209696769714355 cpu‚24.278674225043474 disk.drbd1716.out‚0 disk.sda3.out‚ 1408.89453125 network.sent‚ 3.194365e+06) disk./.usagePercent‚16.31209094092149% disk.nvme0n1.out‚20286.1689453125 disk.sda1.in‚0' disk.nvme0c0n1.out‚20286.1689453125$ memory_percent‚85.47501564025879 network.recv‚533499 disk.sda2.in‚0 disk.rbd0.out‚0 disk.sda3.in‚0# proc.memory.rssMB‚ 2819.63671875 disk.rbd0.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 7.0498046875N.’é ‚(:‡ ºùâÔè¦ý  gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.247' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0494 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0žŠiZê ‚(:Ï ºùâÔìÔ network.sent‚ 3.280736e+06 disk.dm-1.in‚0 disk.dm-1.out‚ 1427.00390625 disk.sda3.in‚0$ memory_percent‚85.47203540802002% disk./.usageGB‚243.06190490722656 disk.sda1.out‚0) proc.memory.availableMB‚ 2380.26171875 disk.drbd2265.out‚0 disk.sda3.out‚ 1427.00390625 disk.rbd1.out‚ 7.3076171875 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 proc.cpu.threads‚17$ disk.nvme0c0n1.out‚ 20286.6171875 disk.dm-0.out‚ 20286.6171875* disk./.usagePercent‚16.312088380846145 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.rbd0.out‚0 network.recv‚538707 disk.dm-0.in‚ 338.32421875 disk.drbd1716.in‚0 disk.sda.out‚ 1427.00390625 disk.drbd2266.out‚0* proc.memory.percent‚17.209696769714355 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0" disk.nvme0n1.out‚ 20286.6171875 disk.drbd2266.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 2819.63671875 cpu‚24.292092634560973%ÄÇ2ë - _wandbruntime‚1468  _runtime‚14683Q¸BÉì à trainer/global_step‚1799 % _timestamp‚1.7883946952474542e+09 _runtime‚1468.227253396 _step‚142 ' lr-AdamW/pg1‚0.00010657256156575476 ' lr-AdamW/pg2‚0.00010657256156575476cU߶Îí È _runtime‚1468.227253396 _step‚142 ' lr-AdamW/pg1‚0.00010657256156575476 ' lr-AdamW/pg2‚0.00010657256156575476 trainer/global_step‚1799 % _timestamp‚1.7883946952474542e+09Ž’¼o2î - _wandbruntime‚1468  _runtime‚1468Õ¹£NÆ'ï ÀO [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1309705282048 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163885568 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚99783756355072 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24947336 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚100977514106368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19525 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23703773 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚98474051073024 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚98474034033664 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1309705282048 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8706830 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659456 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 238505472 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504013 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32964034 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚460 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14996483 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8707287 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111672 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163885568 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚99783575430144 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32964034 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19527 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1365979338752 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 213601792 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1365765736960 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 % _timestamp‚1.7883946952485123e+09 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚95625822545920 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1309541396480 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 trainer/global_step‚1799 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚99611748369408 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1302768340153 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32983559 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 _step‚143 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57930897 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚99611773273088 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚100977752611840 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24947336 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1302608267841 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚99783575430144 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚98474034033664 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚100977752611840 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24947336 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1302608267841 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚99783575430144 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚98474034033664 IDeviceStatsMonitor.on_train_batch_start/allocation.all.curre6«µ"« nt‚19527 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1365979338752 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 213601792 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1365765736960 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 % _timestamp‚1.7883946952485123e+09 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚95625822545920 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23487 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1309541396480 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820805 trainer/global_step‚1799 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚99611748369408 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23487 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1302768340153 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32983559 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19525 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 _step‚143 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180924928 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57930897 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19527 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚99611773273088 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180924928 Ì];2ñ - _wandbruntime‚1468  _runtime‚1468ÚQÐ&ò ÊM O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚98527423955968 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24960832 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1303315227301 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 % _timestamp‚1.7883946959385207e+09 _step‚144 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚99837857037312 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚99837676099072 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚99665775101952 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚96981232608749 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23716485 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚101032496292352 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24960832 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33001577 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1366505504768 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57942858 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚98527440995328 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1310416041984 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚95677740269568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚99665750198272 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57962411 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 215685632 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚96981055496869 trainer/global_step‚1799 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1366721190400 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚98527423955968 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57962411 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57942858 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32982026 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23716015 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8711940 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 240589312 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1310252143104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33001577 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15004545 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚95677757308928 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 _runtime‚1468.918773488 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32982026 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1303475299821é¾dÕ&ó ÏM F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚95677740269568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23487 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚99665750198272 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57962411 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 215685632 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504013 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23764 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659456 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚96981055496869 trainer/global_step‚1799 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1366721190400 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938240 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚98527423955968 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57962411 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57942858 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32982026 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820805 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163898880 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23716015 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8711940 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 240589312 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072520 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23764 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19551 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938240 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1310252143104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33001577 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15004545 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111880 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚95677757308928 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 _runtime‚1468.918773488 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32982026 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1303475299821 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚98527423955968 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19553 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24960832 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1303315227301 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19551 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 % _timestamp‚1.7883946959385207e+09 _step‚144 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚99837857037312 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚99837676099072 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚99665775101952 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚96981232608749 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23716485 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚101032496292352 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24960832 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33001577 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1366505504768 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57942858 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚98527440995328 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1310416041984 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960–Uöä2ô - _wandbruntime‚1468  _runtime‚1468Wo=Tõ Î 1loss/FBt4/assignment_loss‚0.48984231642775977 _step‚145 2loss/SRqqt2/assignment_loss‚2.9936685705184938 0loss/SRqqt4/detection_loss‚2.262149391429765 1loss/SRqqt4/assignment_loss‚2.161518884229136 0loss/FBt2/assignment_loss‚0.2812231374848792 /loss/FBt4/detection_loss‚4.5126022006336015 %loss/total_loss‚56.24331580425663 epoch‚2 trainer/global_step‚1799 0loss/FRt2/assignment_loss‚3.8620169399814173 ,loss/FRt3/detection_loss‚2.2524518703205 .loss/FRt4/detection_loss‚2.536234762271245 1loss/SRqqt1/detection_loss‚1.4828617391623047 /loss/FBt3/detection_loss‚2.5227432055665133 /loss/FRt2/detection_loss‚2.3236552729186686 0loss/FRt1/assignment_loss‚3.4852872327813564 /loss/FRt4/assignment_loss‚4.022942789395651 1loss/SRqqt3/assignment_loss‚1.866946721141287 1loss/SRqqt2/detection_loss‚2.6515408182144165 .loss/FBt1/detection_loss‚3.976824176311493 /loss/FBt2/detection_loss‚1.7134242308918963 _runtime‚1468.920153503 /loss/FRt3/assignment_loss‚4.182013637777688 0loss/FBt3/assignment_loss‚0.3382453123994592 $ _timestamp‚1.788394695940475e+09 2loss/SRqqt1/assignment_loss‚1.7432837782249058 0loss/FBt1/assignment_loss‚0.7122541956392524 .loss/FRt1/detection_loss‚2.251434847712517 1loss/SRqqt3/detection_loss‚1.61814977282242810Ò1”Yö Ó 1loss/SRqqt4/assignment_loss‚2.161518884229136 0loss/FBt2/assignment_loss‚0.2812231374848792 /loss/FBt4/detection_loss‚4.5126022006336015 %loss/total_loss‚56.24331580425663 epoch‚2 trainer/global_step‚1799 0loss/FRt2/assignment_loss‚3.8620169399814173 ,loss/FRt3/detection_loss‚2.2524518703205 .loss/FRt4/detection_loss‚2.536234762271245 1loss/SRqqt1/detection_loss‚1.4828617391623047 /loss/FBt3/detection_loss‚2.5227432055665133 /loss/FRt2/detection_loss‚2.3236552729186686 0loss/FRt1/assignment_loss‚3.4852872327813564 /loss/FRt4/assignment_loss‚4.022942789395651 1loss/SRqqt3/assignment_loss‚1.866946721141287 1loss/SRqqt2/detection_loss‚2.6515408182144165 .loss/FBt1/detection_loss‚3.976824176311493 /loss/FBt2/detection_loss‚1.7134242308918963 _runtime‚1468.920153503 /loss/FRt3/assignment_loss‚4.182013637777688 0loss/FBt3/assignment_loss‚0.3382453123994592 $ _timestamp‚1.788394695940475e+09 2loss/SRqqt1/assignment_loss‚1.7432837782249058 0loss/FBt1/assignment_loss‚0.7122541956392524 .loss/FRt1/detection_loss‚2.251434847712517 1loss/SRqqt3/detection_loss‚1.6181497728224281 1loss/FBt4/assignment_loss‚0.48984231642775977 _step‚145 2loss/SRqqt2/assignment_loss‚2.9936685705184938 0loss/SRqqt4/detection_loss‚2.262149391429765‘B7‘÷ ‚(:† ÉùâÔ”•°  gpu.0.gpu‚56.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚139.155' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.831 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0R'ΤZø ‚(:Ï ÉùâÔ˜ÞÛ ) proc.memory.availableMB‚ 2380.22265625 cpu‚24.305601598936743 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda2.out‚0 disk.nvme0n1.in‚ 338.32421875" disk.nvme0n1.out‚ 20286.7890625% disk./.usageGB‚243.06202697753906 disk.sda.in‚0 disk.sda1.in‚0$ memory_percent‚85.47227382659912# proc.memory.rssMB‚ 2819.63671875* proc.memory.percent‚17.209696769714355 network.sent‚ 3.286108e+06 disk.sda2.in‚0 disk.sda.out‚ 1446.88671875 disk.drbd2266.out‚0* disk./.usagePercent‚16.312096573087242 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.dm-0.in‚ 338.32421875 disk.dm-0.out‚ 20286.7890625 disk.drbd2266.in‚0 disk.dm-1.out‚ 1446.88671875 proc.cpu.threads‚17$ disk.nvme0c0n1.out‚ 20286.7890625 disk.sda1.out‚0 disk.dm-1.in‚0 disk.sda3.in‚0 network.recv‚542159 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd2080.out‚0 disk.rbd1.out‚ 7.3076171875 disk.rbd0.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 1446.88671875¡–ñ‘ù ‚(:† ØùâÔ®ñì  gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.684' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5368 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ôÍÄ\ú ‚(:Ñ ØùâÔõ¿¡ % disk.nvme0n1.out‚20287.4677734375 disk.drbd2265.out‚0 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.sda1.in‚0# proc.memory.rssMB‚ 2819.73828125 proc.cpu.threads‚17) disk./.usagePercent‚16.31211398159957% disk./.usageGB‚243.06228637695312 disk.drbd2265.in‚0 disk.sda2.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.rbd1.out‚ 7.3701171875" disk.nvme0c0n1.in‚ 338.32421875) proc.memory.percent‚17.21031665802002 cpu‚24.3488042117891 disk.rbd1.in‚0 disk.sda.out‚ 1466.1640625 network.sent‚ 3.373899e+06 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.sda1.out‚0 disk.dm-0.in‚ 338.32421875 disk.sda3.in‚0 disk.sda3.out‚ 1466.1640625" disk.dm-0.out‚20287.4677734375 network.recv‚549014 disk.dm-1.out‚ 1466.1640625' disk.nvme0c0n1.out‚20287.4677734375$ memory_percent‚85.47451496124268 disk.drbd2266.out‚0 disk.dm-1.in‚0) proc.memory.availableMB‚ 2379.85546875¿»ø¬‘û ‚(:† çùâÔ˜¼¨  gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.684' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5368 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ï"}zZü ‚(:Ï çùâÔ¿Œµ $ disk./.usageGB‚243.0622673034668 disk.sda3.in‚0 disk.sda2.in‚0 disk.sda.out‚ 1484.6640625 disk.drbd2266.in‚0 disk.sda1.out‚0( proc.memory.availableMB‚ 2379.9140625# proc.memory.rssMB‚ 2819.73828125( disk./.usagePercent‚16.3121127015619 disk.nvme0n1.in‚ 338.32421875 disk.dm-1.in‚0 proc.cpu.threads‚17 network.sent‚ 3.38004e+06' disk.nvme0c0n1.out‚20287.7060546875 disk.drbd1716.in‚0$ memory_percent‚85.47415733337402 disk.sda2.out‚0% disk.nvme0n1.out‚20287.7060546875 disk.drbd2266.out‚0 disk.dm-1.out‚ 1484.6640625 disk.sda3.out‚ 1484.6640625 disk.dm-0.in‚ 338.32421875 disk.rbd0.out‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 7.3701171875" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda1.in‚0) proc.memory.percent‚17.21031665802002" disk.dm-0.out‚20287.7060546875 disk.drbd1716.out‚0 disk.rbd0.in‚0 network.recv‚553129 disk.sda.in‚0 disk.drbd2265.in‚0 cpu‚24.4276114507548563x… 2ý - _wandbruntime‚1514  _runtime‚1514-8ï©A þ Á¥ 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4699985682964325 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.29305553436279297 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9605262875556946 jet/accuracy_6_of_7‚0 particle/accuracy_7_of_7‚0.5 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9901787042617798 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9919354915618896 particle/accuracy_3_of_4‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03960573673248291 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.14652341604232788 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3534438908100128 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9980986714363098 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.10522109270095825 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18673016130924225 particle/accuracy_4_of_5‚1 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/2FRt1SRqqt0F~†/ùBt/FRt_purity‚0.09712210297584534 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879169464111 *jet/accuracy_2_of_3‚0.1933630257844925 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3083333373069763 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.31688931584358215 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3172222077846527 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.3642857074737549 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2391791045665741 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.28022700548171997 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3201833665370941 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6266440749168396 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.04866877570748329 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9930650591850281 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8213859796524048 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10165813565254211 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.020076600834727287 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1558043360710144 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8082175254821777 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9262452721595764 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 jet/accuracy_4_of_7‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 *loss/val_total_loss‚55.884047131184616 8"Purity/0FRt1SRqqt0FBt/event_purity‚0.384216845035553 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3405449390411377 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 jet/accuracy_6_of_6‚0 particle/accuracy_4_of_7‚1 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 -jet/accuracy_4_of_4‚0.0010080644860863686 particle/accuracy_4_of_6‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2524625360965729 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.14548641443252563 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 particle/accuracy_1_of_7‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3549502193927765 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19070662558078766 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.09391321986913681 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.13683129847049713 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.984046995639801 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16105805337429047 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5103042721748352 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9657737016677856 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.10833333432674408 6 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.997067391872406 8"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.384216845035553 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.043283261358737946 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.30000945925712585 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9882266521453857 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.23358583450317383 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.026143791154026985 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 epoch‚2 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9288889169692993 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.23788359761238098 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.28724488615989685 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3385416567325592 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9683594107627869 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3055555522441864 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.09922768920660019 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.375 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.33072590827941895 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.29502683877944946 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4122023582458496 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_5‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.11459660530090332 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3702908158302307 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 _step‚146 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.39918074011802673 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.28518518805503845 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1407216340303421 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0763888955116272 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13050703704357147 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3945310115814209 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.17767857015132904 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_1_of_4‚1 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3452964425086975 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5153239369392395 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3070175349712372 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.35484579205513 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5701754093170166 particle/accuracy_6_of_7‚1 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.06267436593770981 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10353212058544159 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.37147650122642517 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1376066505908966 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_3_of_7‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9839492440223694 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42904502153396606 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4110962152481079 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9928882122039795 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26992419362068176 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4346902668476105 (particle/f_score‚0.33253201842308044 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24725458025932312 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13051949441432953 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 particle/accuracy_2_of_2‚1 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3743816614151001 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.12801650166511536 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.08160918951034546 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08491555601358414 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8395130038261414 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.46268516778945923 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9654629230499268 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 particle/accuracy_5_of_7‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9955989718437195 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.11414426565170288 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4126847982406616 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +jet/accuracy_3_of_3‚0.01207274105399847 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43702179193496704 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.418034166097641 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.06820276379585266 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1428571492433548 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_1_of_1‚1 particle/accuracy_4_of_4‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3622930943965912 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2536134123802185 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4084569811820984 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9925281405448914 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.27170178294181824 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.2971014380455017 *jet/accuracy_1_of_2‚0.6546967029571533 particle/accuracy_2_of_5‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.43409866094589233 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 +jet/accuracy_3_of_5‚0.06382640451192856 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3304162621498108 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.16375434398651123 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.08636276423931122 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5877192616462708 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07829532027244568 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7793846130371094 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0694444477558136 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18533405661582947 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9892371892929077 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3316051959991455 8"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.360636442899704 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_3‚1 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.23544971644878387 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3748686909675598 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.07539684325456619 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.30128204822540283 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_6‚1 particle/accuracy_3_of_6‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.39099863171577454 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006220748648047447 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 jet/accuracy_4_of_5‚0 -"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3125 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.201171413064003 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.003981271293014288 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.41430407762527466 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.234375 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 8"Purity/0FRt*SRqqt0FBt/event_purity‚0.305181086063385 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2791691720485687 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19843856990337372 5 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.14060078561306 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2948717772960663 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07282429188489914 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.25288036465644836 8"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.422839492559433 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9969493746757507 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.31307122111320496 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9879032373428345 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.15219488739967346 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 )jet/accuracy_2_of_5‚0.320027232170105 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4403441548347473 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12475067377090454 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9915422201156616 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.28728070855140686 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.0943707749247551 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.03333333507180214 jet/accuracy_7_of_7‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.27415281534194946 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.46770355105400085 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 +particle/specificity‚0.9474945664405823 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6451758146286011 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3951009511947632 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9879317283630371 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07152702659368515 7 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.1165195181965828 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.983354389667511 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.07648148387670517 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 +jet/accuracy_2_of_2‚0.08074694126844406 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11790475994348526 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9868937134742737 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.007089685183018446 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03721435368061066 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14774034917354584 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1428571492433548 trainer/global_step‚1811 _runtime‚1514.508939991 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2907642126083374 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 jet/accuracy_5_of_6‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9666034579277039 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.33773723244667053 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.10156171023845673 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.2560793161392212 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.24325744807720184 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09876543283462524 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.01344086043536663 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4430006146430969 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0694444477558136 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.32970714569091797 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 +jet/accuracy_2_of_4‚0.25279876589775085 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 % _timestamp‚1.7883947415270345e+09 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.42091619968414307 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1425067037343979 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3629627525806427 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.01388888992369175 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_1_of_5‚0.8680942058563232 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.8947368264198303 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9906010031700134 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.37035268545150757 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014863765798509121 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 6 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.122102752327919 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 jet/accuracy_5_of_7‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.13333334028720856 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.31483587622642517 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.006272401660680771 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 6 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.400661826133728 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.002377206925302744 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.09649123251438141 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.47494515776634216 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3862916827201843 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 particle/accuracy_2_of_7‚1 +"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17163105309009552 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9218108057975769 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001676026382483542 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_2_of_7‚0.5 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08159900456666946 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9915566444396973 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3918202519416809 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.036507654935121536 particle/accuracy_5_of_6‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.00732843205332756 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2637864947319031 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.1606065034866333 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.25520583987236023 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.34286099672317505 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_2‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.40416669845581055 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.10665442794561386 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 jet/accuracy_5_of_5‚0 particle/accuracy_3_of_3‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.32057690620422363 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt*ÒŽ SRqqt0FBt/event_purity‚0 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5000000596046448 *jet/accuracy_2_of_6‚0.2982456386089325 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12783674895763397 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08491555601358414 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.11438591033220291 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08055555820465088 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.006272401660680771 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04012896120548248 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_7‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3205128312110901 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9802340269088745 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6428571343421936 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.34306907653808594 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9275794625282288 particle/accuracy_5_of_5‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_1_of_7‚0.5 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2924021780490875 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.13131314516067505 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.40279319882392883 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03739992156624794 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9973567724227905 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.21536797285079956 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.017029786482453346øÜdvÿ Æ¥ 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.0943707749247551 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.03333333507180214 jet/accuracy_7_of_7‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.27415281534194946 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.46770355105400085 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 +particle/specificity‚0.9474945664405823 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6451758146286011 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3951009511947632 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9879317283630371 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07152702659368515 7 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.1165195181965828 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.983354389667511 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.07648148387670517 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 +jet/accuracy_2_of_2‚0.08074694126844406 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11790475994348526 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9868937134742737 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.007089685183018446 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03721435368061066 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14774034917354584 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1428571492433548 trainer/global_step‚1811 _runtime‚1514.508939991 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.2907642126083374 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 jet/accuracy_5_of_6‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9666034579277039 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.33773723244667053 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.10156171023845673 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.2560793161392212 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.24325744807720184 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09876543283462524 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.01344086043536663 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4430006146430969 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0694444477558136 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.32970714569091797 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 +jet/accuracy_2_of_4‚0.25279876589775085 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 % _timestamp‚1.7883947415270345e+09 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.42091619968414307 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1425067037343979 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3629627525806427 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.01388888992369175 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_1_of_5‚0.8680942058563232 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.8947368264198303 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9906010031700134 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.37035268545150757 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014863765798509121 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 6 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.122102752327919 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 jet/accuracy_5_of_7‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.13333334028720856 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.31483587622642517 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.006272401660680771 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 6 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.400661826133728 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.002377206925302744 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.09649123251438141 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.47494515776634216 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3862916827201843 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 particle/accuracy_2_of_7‚1 +"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.25 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17163105309009552 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9218108057975769 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001676026382483542 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_2_of_7‚0.5 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08159900456666946 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9915566444396973 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3918202519416809 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.036507654935121536 particle/accuracy_5_of_6‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.00732843205332756 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2637864947319031 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.1606065034866333 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.25520583987236023 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.34286099672317505 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_2‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.40416669845581055 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.10665442794561386 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 jet/accuracy_5_of_5‚0 particle/accuracy_3_of_3‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.32057690620422363 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5000000596046448 *jet/accuracy_2_of_6‚0.2982456386089325 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12783674895763397 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08491555601358414 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.11438591033220291 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08055555820465088 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.006272401660680771 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04012896120548248 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_7‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3205128312110901 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9802340269088745 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6428571343421936 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.34306907653808594 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9275794625282288 particle/accuracy_5_of_5‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 jet/accuracy_1_of_7‚0.5 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2924021780490875 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.13131314516067505 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.40279319882392883 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03739992156624794 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9973567724227905 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.21536797285079956 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.017029786482453346 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4699985682964325 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.29305553436279297 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9605262875556946 jet/accuracy_6_of_7‚0 particle/accuracy_7_of_7‚0.5 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9901787042617798 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9919354915618896 particle/accuracy_3_of_4‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03960573673248291 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.14652341604232788 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3534438908100128 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9980986714363098 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.10522109270095825 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18673016130924225 particle/accuracy_4_of_5‚1 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.09712210297584534 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879169464111 *jet/accuracy_2_of_3‚0.1933630257844925 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3083333373069763 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.31688931584358215 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3172222077846527 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.3642857074737549 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2391791045665741 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.28022700548171997 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3201833665370941 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6266440749168396 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.04866877570748329 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9930650591850281 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8213859796524048 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10165813565254211 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.020076600834727287 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1558043360710144 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8082175254821777 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9262452721595764 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 jet/accuracy_4_of_7‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 *loss/val_total_loss‚55.884047131184616 8"Purity/0FRt1SRqqt0FBt/event_purity‚0.384216845035553 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3405449390411377 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 jet/accuracy_6_of_6‚0 particle/accuracy_4_of_7‚1 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 -jet/accuracy_4_of_4‚0.0010080644860863686 particle/accuracy_4_of_6‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2524625360965729 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.14548641443252563 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 particle/accuracy_1_of_7‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3549502193927765 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19070662558078766 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.09391321986913681 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.13683129847049713 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.984046995639801 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16105805337429047 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5103042721748352 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9657737016677856 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.10833333432674408 6 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.997067391872406 8"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.384216845035553 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.043283261358737946 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.30000945925712585 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9882266521453857 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.23358583450317383 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.026143791154026985 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 epoch‚2 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9288889169692993 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.23788359761238098 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.28724488615989685 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3385416567325592 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9683594107627869 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3055555522441864 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.09922768920660019 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.375 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.33072590827941895 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.29502683877944946 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4122023582458496 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_5‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.11459660530090332 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3702908158302307 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 _step‚146 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.39918074011802673 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.28518518805503845 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1407216340303421 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0763888955116272 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13050703704357147 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3945310115814209 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.17767857015132904 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 particle/accuracy_1_of_4‚1 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3452964425086975 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5153239369392395 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3070175349712372 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 7"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.35484579205513 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5701754093170166 particle/accuracy_6_of_7‚1 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.06267436593770981 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.10353212058544159 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.37147650122642517 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1376066505908966 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_3_of_7‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9839492440223694 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42904502153396606 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4110962152481079 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9928882122039795 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26992419362068176 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4346902668476105 (particle/f_score‚0.33253201842308044 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.24725458025932312 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13051949441432953 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 particle/accuracy_2_of_2‚1 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3743816614151001 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.12801650166511536 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.08160918951034546 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08491555601358414 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8395130038261414 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.46268516778945923 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9654629230499268 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 particle/accuracy_5_of_7‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9955989718437195 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.11414426565170288 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4126847982406616 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +jet/accuracy_3_of_3‚0.01207274105399847 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43702179193496704 8"PÆ%¶iurity/0FRt*SRqqt*FBt/event_purity‚0.418034166097641 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.06820276379585266 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1428571492433548 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_1_of_1‚1 particle/accuracy_4_of_4‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3622930943965912 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2536134123802185 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4084569811820984 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9925281405448914 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.27170178294181824 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.2971014380455017 *jet/accuracy_1_of_2‚0.6546967029571533 particle/accuracy_2_of_5‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.43409866094589233 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 +jet/accuracy_3_of_5‚0.06382640451192856 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3304162621498108 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.16375434398651123 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.08636276423931122 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5877192616462708 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07829532027244568 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7793846130371094 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0694444477558136 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18533405661582947 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9892371892929077 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3316051959991455 8"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.360636442899704 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_3‚1 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.23544971644878387 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3748686909675598 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.07539684325456619 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.30128204822540283 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_6‚1 particle/accuracy_3_of_6‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.39099863171577454 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006220748648047447 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 jet/accuracy_4_of_5‚0 -"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3125 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.201171413064003 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.003981271293014288 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.125 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.41430407762527466 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.234375 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 8"Purity/0FRt*SRqqt0FBt/event_purity‚0.305181086063385 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2791691720485687 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19843856990337372 5 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.14060078561306 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2948717772960663 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07282429188489914 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.25288036465644836 8"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.422839492559433 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9969493746757507 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.31307122111320496 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9879032373428345 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.15219488739967346 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 )jet/accuracy_2_of_5‚0.320027232170105 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4403441548347473 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12475067377090454 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9915422201156616 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.28728070855140686 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt2SRqqt0FBt/event_purity‚0’»Û2±‘€ ‚(:† öùâÔÀ’—  gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.684' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5368 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0õ©{wS ‚(:È öùâÔÊá disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.dm-1.out‚ 1503.40234375# proc.memory.rssMB‚ 2819.73828125 disk.dm-1.in‚0 disk.rbd1.out‚ 7.51171875 network.sent‚ 3.385427e+06 disk.sda.out‚ 1503.40234375 disk.drbd2265.out‚0 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 cpu‚24.49769051367312 disk.drbd2265.in‚0 disk.dm-0.out‚ 20288.1796875$ memory_percent‚85.47353744506836% disk./.usageGB‚243.06224822998047 disk.drbd2080.in‚0 disk.rbd0.out‚0$ disk.nvme0c0n1.out‚ 20288.1796875 disk.dm-0.in‚ 338.32421875 proc.cpu.threads‚17 disk.sda3.out‚ 1503.40234375 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.sda1.out‚0' proc.memory.availableMB‚ 2380.015625 disk.sda2.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda3.in‚0 disk.rbd1.in‚0) proc.memory.percent‚17.21031665802002" disk.nvme0n1.out‚ 20288.1796875 network.recv‚556930) disk./.usagePercent‚16.31211142152423 disk.sda2.in‚0 disk.sda.in‚0I'Ô©‚ ‚(:… …úâÔˆþ•  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚35.0 gpu.0.powerWatts‚123.51' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.702 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0=6`ÌSƒ ‚(:È …úâÔ ›°) proc.memory.percent‚17.21034049987793 disk.nvme0n1.in‚ 338.32421875 disk.rbd1.out‚ 7.51171875 disk.sda3.in‚0 disk.dm-0.in‚ 338.32421875 network.sent‚ 3.489617e+06) disk./.usagePercent‚16.31994448404938% disk.nvme0c0n1.out‚20407.36328125 disk.sda.in‚0 disk.sda3.out‚ 1522.8046875 disk.drbd2266.out‚0$ memory_percent‚86.20011806488037 disk.drbd2265.out‚0 disk.rbd0.in‚0 cpu‚24.50819038585396 disk.dm-0.out‚20407.36328125 disk.sda2.out‚0 disk.drbd2080.out‚0 network.recv‚562432 disk.sda1.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 1522.8046875 disk.rbd1.in‚0 disk.sda1.out‚0 disk.drbd2266.in‚0 proc.cpu.threads‚17 disk.drbd2265.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0) proc.memory.availableMB‚ 2260.97265625$ disk./.usageGB‚243.1789665222168 disk.drbd2080.in‚0# disk.nvme0n1.out‚20407.36328125 disk.sda.out‚ 1522.8046875" proc.memory.rssMB‚ 2819.7421875|Óg5œ„ ‚(:‘ ”úâÔëÖì  gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚35.0 gpu.0.powerWatts‚123.221' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.644199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ø •X… ‚(:Í ”úâÔÇ‘­) disk./.usagePercent‚16.31992118736376 disk.dm-0.out‚20527.94140625 disk.sda3.in‚0 disk.sda2.out‚0 disk.rbd0.in‚0) proc.memory.availableMB‚ 2260.47265625 disk.sda3.out‚ 1542.33984375 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.dm-1.out‚ 1542.33984375" proc.memory.rssMB‚ 2819.7421875 disk.dm-0.in‚ 338.32421875 disk.drbd1716.in‚0 disk.sda.out‚ 1542.33984375 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.sda2.in‚0# disk.nvme0n1.out‚20527.94140625 disk.sda.in‚0 disk.drbd2266.in‚0 disk.dm-1.in‚0) proc.memory.percent‚17.21034049987793 network.recv‚567482 disk.rbd1.in‚0 disk.rbd1.out‚ 7.57421875% disk.nvme0c0n1.out‚20527.94140625 disk.drbd2080.in‚0 disk.drbd2265.out‚0 proc.cpu.threads‚17% disk./.usageGB‚243.17861938476562" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd1716.out‚0 disk.drbd2266.out‚0$ memory_percent‚86.20316982269287 cpu‚24.518690266993698 network.sent‚ 3.494106e+06 disk.nvme0n1.in‚ 338.32421875 disk.sda1.out‚0Äï£i2† - _wandbruntime‚1545  _runtime‚1545å¢c¸É‡ à _step‚147 ' lr-AdamW/pg1‚0.00010544557118464838 ' lr-AdamW/pg2‚0.00010544557118464838 trainer/global_step‚1849 % _timestamp‚1.7883947727629685e+09 _runtime‚1545.742760873XPîΈ È ' lr-AdamW/pg2‚0.00010544557118464838 trainer/global_step‚1849 % _timestamp‚1.7883947727629685e+09 _runtime‚1545.742760873 _step‚147 ' lr-AdamW/pg1‚0.00010544557118464838“!C.Ò2‰ - _wandbruntime‚1545  _runtime‚1545ÌVÂÒ'Š ÌO ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚102320265651200 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1359775363584 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚103680041014784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 trainer/global_step‚1849 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚104924074912256 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚102320248611840 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1352442016742 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚34098139 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚103680041014784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚449 % _timestamp‚1.7883947727639809e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚34098139 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1417580006912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24693121 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 201018368 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚102320248611840 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1359775363584 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _step‚148 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚103506519809024 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25945716 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚103679860089344 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1359611477504 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9050320 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚103679860089344 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚102320265651200 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1359611477504 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15642352 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚34117665 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99299848865792 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25945714 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24692669 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 225922048 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100652290882534 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15642349 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99299865905152 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚60043853 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚34117665 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1417781025280 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0cê×Y´'‹ ÑO RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚103679860089344 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚102320265651200 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1359611477504 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15642352 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚34117665 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99299848865792 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25945714 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24692669 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 225922048 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100652290882534 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15642349 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99299865905152 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚60043853 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚34117665 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1417781025280 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚102320265651200 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1359775363584 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚103680041014784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 trainer/global_step‚1849 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚104924074912256 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚102320248611840 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1352442016742 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚34098139 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚103680041014784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚449 % _timestamp‚1.7883947727639809e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚34098139 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1417580006912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24693121 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 201018368 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚102320248611840 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1359775363584 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _step‚148 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚103506519809024 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25945716 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚103679860089344 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1359611477504 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9050320 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_byteÙôê#s.large_pool.peak‚ 9141485568”ÍÆ%;2Œ - _wandbruntime‚1546  _runtime‚1546H‚Æ`Ü& ÖM SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚102373655573504 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1418531920896 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚60075341 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚100705092677526 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚103560521637888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 trainer/global_step‚1849 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1360322224128 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209393664 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 % _timestamp‚1.7883947734472263e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34116131 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚34116131 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚453 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 _step‚149 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚60094895 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234297344 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚99351783628800 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25959212 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1360486123520 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1418322527232 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25959210 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1353148976202 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚103733960758272 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _runtime‚1546.427479541 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚60094895 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34135683 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104979053558784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚103734141697024 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚34135683 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24705255 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25959210 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚104978819261440 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24705708 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚100705092677526 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚103560521637888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 trainer/global_step‚1849 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1360322224128 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 209393664 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 % _timestamp‚1.7883947734472263e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34116131 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚34116131 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚453 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 _step‚149 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚60094895 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 234297344 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚99351783628800 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25959212 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1360486123520 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1418322527232 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25959210 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1353148976202 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚103733960758272 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _runtime‚1546.427479541 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚60094895 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34135683 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104979053558784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚103734141697024 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚34135683 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24705255 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25959210 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚104978819261440 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24705708 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BtÔ`‘ Ú %loss/total_loss‚56.93959085078587 trainer/global_step‚1849 /loss/FRt3/assignment_loss‚4.021917411978816 1loss/SRqqt1/detection_loss‚1.5491050207877861 0loss/SRqqt3/detection_loss‚1.621117969151257 _step‚150 1loss/SRqqt4/detection_loss‚2.8147757183760405 .loss/FBt3/detection_loss‚2.983042638153118 % _timestamp‚1.7883947734491682e+09 _runtime‚1546.428769025 /loss/FRt1/detection_loss‚2.2001948074570725 /loss/FRt2/detection_loss‚2.3425544919002625 0loss/FRt2/assignment_loss‚3.4805582336017062 2loss/SRqqt1/assignment_loss‚1.9698615180985892 2loss/SRqqt2/assignment_loss‚2.7392751719500565 1loss/FBt2/assignment_loss‚0.22168131930130397 .loss/FRt3/detection_loss‚2.402849559632825 /loss/FBt2/detection_loss‚1.3259638073982603 0loss/FRt1/assignment_loss‚3.6274403227227077 /loss/FRt4/assignment_loss‚4.516739761545544 /loss/FRt4/detection_loss‚2.6286247869332633 /loss/FBt4/detection_loss‚3.5226507738784507 epoch‚3 0loss/FBt1/assignment_loss‚0.4358455552623387 .loss/FBt1/detection_loss‚4.561699028526034 1loss/SRqqt2/detection_loss‚3.0987312535981872 1loss/FBt4/assignment_loss‚0.37853892151179025 2loss/SRqqt4/assignment_loss‚2.3205555570311844 2loss/SRqqt3/assignment_loss‚1.8098888321190465 0loss/FBt3/assignment_loss‚0.3659783898702268–¨R’’ ‚(:‡ £úâÔ§¶Æ  gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚164.487' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.8974 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0–&á¢[“ ‚(:Ð £úâÔ‹¡ê  network.sent‚ 3.500338e+06" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd2080.in‚0 disk.sda.out‚ 1560.7265625 disk.dm-1.out‚ 1560.7265625$ memory_percent‚86.20495796203613 disk.rbd0.out‚0* disk./.usagePercent‚16.319929379604858 disk.rbd1.in‚0 disk.dm-0.in‚ 338.32421875" disk.dm-0.out‚20528.2802734375 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 338.32421875 cpu‚24.529227781800653 proc.cpu.threads‚17 disk.sda.in‚0 disk.drbd2266.out‚0 disk.sda2.in‚0% disk./.usageGB‚243.17874145507812 disk.rbd1.out‚ 7.57421875 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.sda1.in‚0( proc.memory.availableMB‚ 2260.1796875) proc.memory.percent‚17.21034049987793 disk.drbd1716.in‚0 disk.sda3.in‚0 disk.sda1.out‚0 disk.rbd0.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0" proc.memory.rssMB‚ 2819.7421875 network.recv‚571870' disk.nvme0c0n1.out‚20528.2802734375 disk.drbd1716.out‚0% disk.nvme0n1.out‚20528.2802734375 disk.sda3.out‚ 1560.7265625 disk.sda2.out‚0“J5Š” ‚(:„ ²úâÔ‚⃠ gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚176.15' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.23 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚01ûæ&Y• ‚(:Î ²úâÔãäÌ disk.dm-1.out‚ 1578.3984375 disk.nvme0n1.in‚ 338.32421875 disk.rbd1.out‚ 7.7158203125" disk.nvme0c0n1.in‚ 338.32421875% proc.memory.availableMB‚ 2260.6875 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.sda2.in‚0 disk.drbd2266.out‚0) proc.memory.percent‚17.21034049987793" disk.dm-0.out‚20528.9521484375 disk.sda2.out‚0 disk.sda1.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 1578.3984375 disk.rbd0.out‚0 network.sent‚ 3.586759e+06 network.recv‚577184* disk./.usagePercent‚16.319929635612393$ disk./.usageGB‚243.1787452697754 disk.drbd1716.in‚0 disk.drbd2265.in‚0 disk.dm-0.in‚ 338.32421875 disk.drbd1716.out‚0% disk.nvme0n1.out‚20528.9521484375 disk.drbd2265.out‚0 disk.dm-1.in‚0' disk.nvme0c0n1.out‚20528.9521484375$ memory_percent‚86.20185852050781 cpu‚24.539597885562273 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.sda1.in‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda.out‚ 1578.3984375" proc.memory.rssMB‚ 2819.7421875¨zÂx2– - _wandbruntime‚1584  _runtime‚1584PŒõÉ— à ' lr-AdamW/pg1‚0.00010428078843653421 ' lr-AdamW/pg2‚0.00010428078843653421 trainer/global_step‚1899 % _timestamp‚1.7883948113738952e+09 _runtime‚1584.353691323 _step‚1516éãΘ È ' lr-AdamW/pg1‚0.00010428078843653421 ' lr-AdamW/pg2‚0.00010428078843653421 trainer/global_step‚1899 % _timestamp‚1.7883948113738952e+09 _runtime‚1584.353691323 _step‚151—MÖOÃ2™ - _wandbruntime‚1584  _runtime‚15845®¬,Áš ÎO TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35020617 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35001091 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1395340523008 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25330483 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25329999 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16045299 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9284700 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1395340523008 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1395176636928 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚106384921363968 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9285181 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚107661419399168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _step‚152 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 % _timestamp‚1.7883948113749292e+09 _runtime‚1584.358238918 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚106384921363968 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚107661660001280 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚103283729321618 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61621605 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚481 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚101895752087552 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚106206586348544 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106206611252224 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26620516 \HDeviceStatsMonitor.on_train_batch_start/allocat¦-Y×ed_bytes.large_pool.freed‚104989744727040 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚104989761766400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚26620514 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1387817161750 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26620516 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚106385102289408 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61641133 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16045302 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚103283552209942 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚104989761766400 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35001091 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9285181 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚107661419399168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _step‚152 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 % _timestamp‚1.7883948113749292e+09 _runtime‚1584.358238918 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚106384921363968 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚107661660001280 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚103283729321618 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61621605 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚481 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚101895752087552 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚106206586348544 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106206611252224 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26620516 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚104989744727040 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚104989761766400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚26620514 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1387817161750 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26620516 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚106385102289408 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61641133 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16045302 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚103283552209942 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚104989761766400 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35001091 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 _step‚153 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9289381 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 228005888 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26634012 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25342739 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚103336176893050 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61672647 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚103336354004934 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1388524121210 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35019083 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚106439022032896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1395887383552 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26634010 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚107716173480448 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61653093 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚106439202971648 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35038635 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106260613081088 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16053358AÓNá!ž ÝM RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9289827 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26634012 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25342739 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚103336176893050 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61672647 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚103336354004934 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1388524121210 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35019083 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚106439022032896 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1395887383552 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26634010 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚107716173480448 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61653093 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚106439202971648 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35038635 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106260613081088 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16053358 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 _runtime‚1585.046592415 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35019083 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚106439202971648 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚449 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 _step‚153 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9289381 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 228005888 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 PDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚109090163564032 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚109089982638592 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9518960 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1492252868096 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27295316 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1430905682432 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25967685 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35904043 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _step‚156 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚109090163564032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27295316 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35923569 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883948499393828e+09 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1430741796352 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27295314 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25967209 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63199357 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚1949 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚107659257881600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032ÎÛgÙ'­ ÓO L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27295314 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25967209 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63199357 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚1949 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚107659257881600 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚109089982638592 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚473 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚104491638269952 _runtime‚1622.919409751 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚107659240842240 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚110398955563520 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚107659257881600 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚27295314 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚104491621230592 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16448249 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚105914813537350 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35904043 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1430741796352 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16448252 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9519433 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚108906702695424 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1430905682432 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚105914990649026 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚110398714961408 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1492037169664 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63218885 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚107659240842240 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚109090163564032 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚109089982638592 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9518960 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1492252868096 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27295316 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1430905682432 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25967685 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35904043 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _step‚156 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚109090163564032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27295316 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35923569 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883948499393828e+09 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1430741796352 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0œصSÔ2® - _wandbruntime‚1623  _runtime‚1623ïŸkò ¯ ×M VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚16456311 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚109144264246272 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9523596 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚108960704524288 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238491648 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1492992851968 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35941587 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35922035 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚109144264246272 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63250399 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚104543555993600 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚107712647803904 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1431616442368 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25979904 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚157 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚25980374 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚108960679620608 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63250399 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27308812 MCDeviceStatsMonitor.on_train_batch_end/allocation.sr¥µ÷ëmall_pool.current‚19552 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 213587968 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 trainer/global_step‚1949 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63230845 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1431616442368 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1431452542976 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚109144083307520 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27308810 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27308810 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚104543538954240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1431452542976 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1492779264000 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚105967615332342 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35941587 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚110453458884608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚107712647803904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1424059338742 $ _timestamp‚1.788394850615167e+09 _runtime‚1623.595437519 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35922035 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚109144264246272 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63250399 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚104543555993600 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚107712647803904 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1431616442368 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25979904 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚157 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚25980374 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚108960679620608 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63250399 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27308812 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 213587968 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 trainer/global_step‚1949 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63230845 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1431616442368 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1431452542976 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚109144083307520 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27308810 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27308810 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚104543538954240 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1431452542976 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1492779264000 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚105967615332342 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35941587 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚110453458884608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚107712647803904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1424059338742 $ _timestamp‚1.788394850615167e+09 _runtime‚1623.595437519 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0@Åa÷2± - _wandbruntime‚1623  _runtime‚1623BØeoW² Ñ /loss/FRt2/detection_loss‚2.2072064919242016 /loss/FRt4/assignment_loss‚4.624587410307945 .loss/FRt4/detection_loss‚2.515532039898507 1loss/SRqqt1/detection_loss‚1.6036816629416801 0loss/SRqqt3/detection_loss‚1.886023274844601 % _timestamp‚1.7883948506170676e+09 _runtime‚1623.596705484 /loss/FRt1/assignment_loss‚3.766785875207088 0loss/SRqqt4/detection_loss‚2.492802073826661 /loss/FRt3/assignment_loss‚4.425846449426703 2loss/SRqqt1/assignment_loss‚1.8384106687081405 2loss/SRqqt3/assignment_loss‚1.7356737084568492 2loss/SRqqt4/assignment_loss‚2.4640466655428344 0loss/FBt1/assignment_loss‚0.5363788048637687 -loss/FRt1/detection_loss‚2.05983549258748 1loss/SRqqt2/detection_loss‚2.9281995716420086 trainer/global_step‚1949 .loss/FBt3/detection_loss‚3.171658921279703 0loss/FBt3/assignment_loss‚0.5408522961635955 epoch‚3 0loss/FRt2/assignment_loss‚3.8708591386675835 2loss/SRqqt2/assignment_loss‚3.0918106816031714 /loss/FRt3/detection_loss‚2.4593129049281814 /loss/FBt4/detection_loss‚3.9744900107383727 /loss/FBt2/detection_loss‚1.0199905280807813 _step‚158 1loss/FBt2/assignment_loss‚0.24012924973252872 0loss/FBt4/assignment_loss‚0.4179239941733613 .loss/FBt1/detection_loss‚4.099686182462252 &loss/total_loss‚57.971724098007996þªÊ/\³ Ö 0loss/FBt3/assignment_loss‚0.5408522961635955 epoch‚3 0loss/FRt2/assignment_loss‚3.8708591386675835 2loss/SRqqt2/assignment_loss‚3.0918106816031714 /loss/FRt3/detection_loss‚2.4593129049281814 /loss/FBt4/detection_loss‚3.9744900107383727 /loss/FBt2/detection_loss‚1.0199905280807813 _step‚158 1loss/FBt2/assignment_loss‚0.24012924973252872 0loss/FBt4/assignment_loss‚0.4179239941733613 .loss/FBt1/detection_loss‚4.099686182462252 &loss/total_loss‚57.971724098007996 /loss/FRt2/detection_loss‚2.2072064919242016 /loss/FRt4/assignment_loss‚4.624587410307945 .loss/FRt4/detection_loss‚2.515532039898507 1loss/SRqqt1/detection_loss‚1.6036816629416801 0loss/SRqqt3/detection_loss‚1.886023274844601 % _timestamp‚1.7883948506170676e+09 _runtime‚1623.596705484 /loss/FRt1/assignment_loss‚3.766785875207088 0loss/SRqqt4/detection_loss‚2.492802073826661 /loss/FRt3/assignment_loss‚4.425846449426703 2loss/SRqqt1/assignment_loss‚1.8384106687081405 2loss/SRqqt3/assignment_loss‚1.7356737084568492 2loss/SRqqt4/assignment_loss‚2.4640466655428344 0loss/FBt1/assignment_loss‚0.5363788048637687 -loss/FRt1/detection_loss‚2.05983549258748 1loss/SRqqt2/detection_loss‚2.9281995716420086 trainer/global_step‚1949 .loss/FBt3/detection_loss‚3.171658921279703ž\8®’´ ‚(:‡ îúâÔ˜½º  gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚168.553' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.7106 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0–ÛOå`µ ‚(:Õ îúâÔåç disk.nvme0n1.in‚ 338.32421875$ disk.nvme0n1.out‚20530.517578125% disk./.usageGB‚243.17873764038086 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 1652.90625 disk.dm-0.in‚ 338.32421875 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1652.90625 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.drbd2266.in‚0) proc.memory.percent‚17.21034049987793 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875! disk.dm-0.out‚20530.517578125 disk.drbd1716.in‚0' proc.memory.availableMB‚ 2260.578125 disk.rbd0.out‚0.078125 disk.rbd1.out‚ 8.044921875 network.sent‚ 3.692125e+06 network.recv‚596612* disk./.usagePercent‚16.319929123597323 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd0.in‚ 0.0078125 disk.sda.in‚0" proc.memory.rssMB‚ 2819.7421875 proc.cpu.threads‚17& disk.nvme0c0n1.out‚20530.517578125 disk.drbd2080.out‚0 disk.sda.out‚ 1652.90625 disk.drbd2266.out‚0# memory_percent‚86.2025260925293 disk.sda2.out‚0 cpu‚24.579255532860646çò’¶ ‚(:‡ ýúâÔ†ÿþ  gpu.0.gpu‚48.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚176.114' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.2228 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0B|ÌÃh· ‚(:Ý ýúâÔÕñÕ  disk.sda.out‚ 1667.58984375 disk.sda1.in‚0 disk.rbd1.out‚ 8.130859375" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875$ disk.nvme0n1.out‚20530.708984375 disk.sda3.out‚ 1667.58984375 disk.sda2.out‚0 cpu‚24.58883238862739 disk.sda.in‚0$ memory_percent‚86.20405197143555 proc.cpu.threads‚17 network.sent‚ 3.77873e+06 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0" proc.memory.rssMB‚ 2819.7421875) proc.memory.percent‚17.21034049987793& disk.nvme0c0n1.out‚20530.708984375 disk.sda3.in‚0 disk.sda2.in‚0 network.recv‚601692 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2265.in‚0 disk.dm-1.out‚ 1667.58984375 disk.rbd1.in‚0! disk.dm-0.out‚20530.708984375' proc.memory.availableMB‚ 2260.328125* disk./.usagePercent‚16.319929123597323 disk.sda1.out‚0 disk.drbd1716.out‚0% disk./.usageGB‚243.17873764038086 disk.drbd1716.in‚0 disk.rbd0.out‚0.078125RQ‹‹2¸ - _wandbruntime‚1661  _runtime‚1661û\›Ûɹ à trainer/global_step‚1999 % _timestamp‚1.7883948884254048e+09 _runtime‚1661.405184766 _step‚159 ' lr-AdamW/pg1‚0.00010184176663469014 ' lr-AdamW/pg2‚0.00010184176663469014Ÿp•Îº È % _timestamp‚1.7883948884254048e+09 _runtime‚1661.405184766 _step‚159 ' lr-AdamW/pg1‚0.00010184176663469014 ' lr-AdamW/pg2‚0.00010184176663469014 trainer/global_step‚1999Ÿ&&Á2» - _wandbruntime‚1661  _runtime‚1661I^ %¼ ÎO eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚111606794138624 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1466306955776 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚111795043913216 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1466470841856 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚488 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚485 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108546074864758 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚107087507412992 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64777109 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64796637 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27970116 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚108546251976434 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16851199 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚111795224838656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27970114 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1466470841856 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚110328736957440 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1529426331648 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚113136220470272 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚107087524452352 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚110328753996800 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 238504960 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚111606769234944 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1466306955776 ‹Î´?ÏBDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1466306955776 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚111795043913216 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1466470841856 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚488 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚485 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108546074864758 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚107087507412992 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64777109 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64796637 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27970116 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚108546251976434 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16851199 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚111795224838656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27970114 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1466470841856 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚110328736957440 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1529426331648 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚113136220470272 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚107087524452352 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 Ê£(2¾ - _wandbruntime‚1662  _runtime‚1662ÎHBÞ&¿ ØM KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚110382143919104 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27983610 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207296512 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 trainer/global_step‚1999 _runtime‚1662.093120685 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚110382143919104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36844539 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26617374 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚111849325520896 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27983612 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36844539 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26616912 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚459 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1459434483750 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚110382126879744 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚64808597 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9758113 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1530169711104 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚111660771063808 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16859258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108598876659750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚111660795967488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚113190733478400 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚64828151 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1467017702400 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1467181601792 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64828151 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27983612 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1529962414592 _step‚161 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚111849144582144 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36824987 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚107139425136640 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392Iƒã&À ÝM QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚111660795967488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚113190733478400 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚64828151 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1467017702400 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1467181601792 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64828151 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27983612 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1529962414592 _step‚161 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚111849144582144 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36824987 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚107139425136640 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚110382143919104 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27983610 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 207296512 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 trainer/global_step‚1999 _runtime‚1662.093120685 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚110382143919104 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36844539 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26617374 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚111849325520896 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27983612 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36844539 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26616912 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚459 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1459434483750 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚110382126879744 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚64808597 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9758113 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1530169711104 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚111660771063808 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16859258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108598876659750 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2¡ÞIî2Á - _wandbruntime‚1662  _runtime‚1662§ýfY Ó 1loss/SRqqt1/detection_loss‚1.7495678641729884 0loss/FBt1/assignment_loss‚0.9068373179761693 .loss/FRt1/detection_loss‚2.320285312831402 0loss/SRqqt3/detection_loss‚1.552640588091524 1loss/SRqqt2/assignment_loss‚2.584519127319599 0loss/FBt4/assignment_loss‚0.5518378457286067 /loss/FRt4/detection_loss‚2.4197192935660334 /loss/FRt4/assignment_loss‚4.539180381463305 2loss/SRqqt3/assignment_loss‚1.7066551120889266 1loss/SRqqt4/detection_loss‚2.4916127551543084 epoch‚3 _runtime‚1662.094583176 2loss/SRqqt1/assignment_loss‚1.7682165590977228 1loss/SRqqt2/detection_loss‚3.4079984991714873 trainer/global_step‚1999 _step‚162 0loss/FRt2/assignment_loss‚3.6493743194474115 /loss/FRt2/detection_loss‚2.2738562919199468 /loss/FBt3/detection_loss‚3.1400746721558903 /loss/FRt3/assignment_loss‚4.175442551438873 .loss/FBt1/detection_loss‚4.117912092379162 /loss/FBt2/detection_loss‚1.5594150606955623 % _timestamp‚1.7883948891149685e+09 0loss/FBt2/assignment_loss‚0.3344021960256214 /loss/FRt3/detection_loss‚2.2455796714167335 /loss/FBt4/detection_loss‚3.4371249629901004 &loss/total_loss‚57.744215073815916 /loss/FRt1/assignment_loss‚3.920868532732129 2loss/SRqqt4/assignment_loss‚2.1929742976238855 0loss/FBt3/assignment_loss‚0.6981197683285245æÝÜ¢à Ø 0loss/FBt2/assignment_loss‚0.3344021960256214 /loss/FRt3/detection_loss‚2.2455796714167335 /loss/FBt4/detection_loss‚3.4371249629901004 &loss/total_loss‚57.744215073815916 /loss/FRt1/assignment_loss‚3.920868532732129 2loss/SRqqt4/assignment_loss‚2.1929742976238855 0loss/FBt3/assignment_loss‚0.6981197683285245 1loss/SRqqt1/detection_loss‚1.7495678641729884 0loss/FBt1/assignmeÉ{ Ó¼nt_loss‚0.9068373179761693 .loss/FRt1/detection_loss‚2.320285312831402 0loss/SRqqt3/detection_loss‚1.552640588091524 1loss/SRqqt2/assignment_loss‚2.584519127319599 0loss/FBt4/assignment_loss‚0.5518378457286067 /loss/FRt4/detection_loss‚2.4197192935660334 /loss/FRt4/assignment_loss‚4.539180381463305 2loss/SRqqt3/assignment_loss‚1.7066551120889266 1loss/SRqqt4/detection_loss‚2.4916127551543084 epoch‚3 _runtime‚1662.094583176 2loss/SRqqt1/assignment_loss‚1.7682165590977228 1loss/SRqqt2/detection_loss‚3.4079984991714873 trainer/global_step‚1999 _step‚162 0loss/FRt2/assignment_loss‚3.6493743194474115 /loss/FRt2/detection_loss‚2.2738562919199468 /loss/FBt3/detection_loss‚3.1400746721558903 /loss/FRt3/assignment_loss‚4.175442551438873 .loss/FBt1/detection_loss‚4.117912092379162 /loss/FBt2/detection_loss‚1.5594150606955623 % _timestamp‚1.7883948891149685e+09¢w’­»Ä ‚(:’ ŒûâÔ‚¶¸  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚174.549' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.909800000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0—á+fÅ ‚(:Û ŒûâÔóÖç  disk.dm-1.out‚ 1688.33984375 disk.sda3.out‚ 1688.33984375# disk.nvme0c0n1.out‚ 20531.359375 disk.drbd2080.out‚0 disk.sda.in‚0 disk.drbd2265.out‚0) proc.memory.percent‚17.21034049987793 disk.dm-0.in‚ 338.32421875 disk.sda2.out‚0 disk.drbd1716.out‚0! disk.nvme0n1.out‚ 20531.359375$ memory_percent‚86.47336959838867 network.sent‚ 3.785177e+06 disk.sda3.in‚0 disk.rbd1.out‚ 8.193359375 disk.drbd2265.in‚0 disk.sda1.in‚0 disk.dm-0.out‚ 20531.359375 disk.drbd2080.in‚0 proc.cpu.threads‚17 disk.rbd0.in‚ 0.0078125 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.sda1.out‚0" proc.memory.rssMB‚ 2819.7421875 cpu‚24.598094046529557 network.recv‚606332 disk.drbd1716.in‚0 disk.rbd0.out‚ 0.14111328125* disk./.usagePercent‚16.319932963710336% disk./.usageGB‚243.17879486083984 disk.nvme0n1.in‚ 338.32421875 disk.sda.out‚ 1688.33984375 disk.dm-1.in‚0 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.in‚0' proc.memory.availableMB‚ 2216.203125(…³Æ ‚(:’ ›ûâÔº¶†  gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚35.0 gpu.0.powerWatts‚125.402' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.080400000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0@U“ú_Ç ‚(:Ô ›ûâÔÚÑâ disk.nvme0n1.out‚ 20531.625 disk.sda3.out‚ 1706.07421875" disk.nvme0c0n1.in‚ 338.32421875 disk.nvme0c0n1.out‚ 20531.625 disk.dm-0.out‚ 20531.625) proc.memory.availableMB‚ 2240.44921875 network.sent‚ 3.871726e+06 disk.nvme0n1.in‚ 338.32421875 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 network.recv‚612230% disk./.usageGB‚243.17873764038086 disk.sda.in‚0 disk.drbd2266.out‚0 disk.sda3.in‚0 disk.rbd1.out‚ 8.193359375 disk.dm-0.in‚ 338.32421875$ memory_percent‚86.32538318634033* disk./.usagePercent‚16.319929123597323 disk.rbd0.in‚ 0.0078125 disk.sda1.out‚0 disk.dm-1.out‚ 1706.07421875) proc.memory.percent‚17.21034049987793 proc.cpu.threads‚17 disk.sda.out‚ 1706.07421875 disk.drbd2265.in‚0 disk.dm-1.in‚0" proc.memory.rssMB‚ 2819.7421875 disk.rbd0.out‚ 0.14111328125 disk.sda1.in‚0 disk.drbd1716.out‚0 disk.drbd2266.in‚0 disk.sda2.in‚0 cpu‚24.607455127493715 disk.sda2.out‚0 disk.drbd2080.in‚0×?šgÈ ‚(:’ ªûâÔÕ¦¥  gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚128.064' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.612799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ÓìütÉ ‚(:é ªûâÔÄ‚ disk.dm-0.in‚ 338.32421875 disk.sda.out‚ 1723.80078125" proc.memory.rssMB‚ 2819.7421875 disk.drbd2265.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd1716.in‚0) proc.memory.percent‚17.21034049987793 cpu‚24.6161356328289 disk.rbd0.out‚ 0.2041015625& disk.nvme0n1.out‚20532.13427734375% disk./.usageGB‚243.17871856689453 disk.drbd2265.out‚0 disk.rbd1.out‚ 8.349609375 disk.sda2.in‚0 disk.drbd1716.out‚0* disk./.usagePercent‚16.319927843559654 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd2080.in‚0$ memory_percent‚86.91279888153076) proc.memory.availableMB‚ 2144.20703125 network.sent‚ 3.877203e+06 disk.rbd1.in‚0 disk.rbd0.in‚ 0.0078125 disk.sda.in‚0 proc.cpu.threads‚17 disk.sda3.in‚0 disk.drbd2080.out‚0 network.recv‚616020 disk.dm-1.out‚ 1723.80078125 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.dm-1.in‚0 disk.sda3.out‚ 1723.80078125( disk.nvme0c0n1.out‚20532.13427734375# disk.dm-0.out‚20532.13427734375 disk.drbd2266.in‚0¥Ýk2Ê - _wandbruntime‚1699  _runtime‚1699hVÐ_ÉË Ã ' lr-AdamW/pg1‚0.00010056956399611742 ' lr-AdamW/pg2‚0.00010056956399611742 trainer/global_step‚2049 % _timestamp‚1.7883949270131524e+09 _runtime‚1699.992943338 _step‚1631Ï;ÖÎÌ È ' lr-AdamW/pg2‚0.00010056956399611742 trainer/global_step‚2049 % _timestamp‚1.7883949270131524e+09 _runtime‚1699.992943338 _step‚163 ' lr-AdamW/pg1‚0.00010056956399611742£^pq2Í - _wandbruntime‚1699  _runtime‚1699ÒÕ¥%Ô'Î ÎO IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚111177513303842 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚445 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27240471 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28644914 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9986322 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚109683410634752 % _timestamp‚1.7883949270143328e+09 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27240916 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚37729473 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚114500286113280 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚112998233072640 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚442 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚112998250112000 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17254149 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚112998250112000 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234310656 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1493942596774 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37709947 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1566629829120 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1566420422144 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66374389 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚114306885581824 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚109683393595392 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1501872115200 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚115873515410944 _runtime‚1699.994401811 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28644916 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚115873281100288 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66354861 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1502036001280 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28644914 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1502036001280 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28644916 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114500105187840 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚114500105187840 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440±i[ Ù'Ï ÓO VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚445 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27240471 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28644914 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9986322 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚109683410634752 % _timestamp‚1.7883949270143328e+09 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27240916 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚37729473 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚114500286113280 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚112998233072640 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚442 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚112998250112000 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17254149 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚112998250112000 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 234310656 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1493942596774 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37709947 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1566629829120 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1566420422144 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66374389 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚114306885581824 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚109683393595392 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1501872115200 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚115873515410944 _runtime‚1699.994401811 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28644916 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚115873281100288 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66354861 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1502036001280 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28644914 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1502036001280 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28644916 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114500105187840 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚114500105187840 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚111177513303842¤§^ãk2Ð - _wandbruntime‚1700  _runtime‚1700ãhÅYñÑ ×M NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28658412 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37727939 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66405903 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66386349 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9990944 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _runtime‚ 1700.67924113 _step‚165 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113051622994944 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27253152 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1494809628758 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1494649556234 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚455 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚113051622994944 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚114360887410688 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚113051640034304 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17262208 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 % _timestamp‚1.7883949276991057e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28658410 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27253610 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚111229960875274 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37747491 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚111230137987158 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1502582861824 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1567377204224 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37747491 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1502582861824 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚109735328358400 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032A!¯â&Ò ÜM SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚113051622994944 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚114360887410688 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚113051640034304 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17262208 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 % _timestamp‚1.7883949276991057e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28658410 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27253610 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚111229960875274 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37747491 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚111230137987158 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1502582861824 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1567377204224 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37747491 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1502582861824 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚109735328358400 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28658412 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37727939 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66405903 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66386349 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9990944 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _runtime‚ 1700.67924113 _step‚165 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113051622994944 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27253152 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1494809628758 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1494649556234 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚455 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960¥Ó)z!2Ó - _wandbruntime‚1700  _runtime‚1700Z(ÿVÔ Ð .loss/FRt1/detection_loss‚2.353811801298634 2loss/SRqqt4/assignment_loss‚2.4582264605495663 0loss/FBt1/assignment_loss‚0.4489427070261627 1loss/SRqqt1/detection_loss‚1.7368262775005803 /loss/FBt2/detection_loss‚1.0702564530061551 0loss/FBt2/assignment_loss‚0.2779080953348378 -loss/FRt3/detection_loss‚2.20647900950077 0loss/FBt4/assignment_loss‚0.5449609215417391 2loss/SRqqt1/assignment_loss‚1.8934053466433571 2loss/SRqqt2/assignment_loss‚2.6865106503168743 1loss/FBt3/assignment_loss‚0.43017609900871995 0loss/SRqqt2/detection_loss‚3.373492936293284 0loss/SRqqt3/assignment_loss‚1.82168982187286 &loss/total_loss‚55.411418681762065 /loss/FRt2/assignment_loss‚4.230533777034446 /loss/FRt3/assignment_loss‚4.235580192055813 /loss/FRt4/assignment_loss‚4.605956085990457 .loss/FRt2/detection_loss‚2.618914145006709 1loss/SRqqt4/detection_loss‚2.2353732655445735 epoch‚3 /loss/FBt1/detection_loss‚3.1591909304261208 /loss/FBt4/detection_loss‚3.2877504891819425 trainer/global_step‚2049 _runtime‚1700.680280803 0loss/FRt1/assignment_loss‚3.4719015759426157 .loss/FRt4/detection_loss‚2.517671910804861 1loss/SRqqt3/detection_loss‚1.4150126394815743 /loss/FBt3/detection_loss‚2.3308470903994127 % _timestamp‚1.7883949277006633e+09 _step‚1661AdÀ[Õ Õ 0loss/SRqqt3/assignment_loss‚1.82168982187286 &loss/total_loss‚55.411418681762065 /loss/FRt2/assignment_loss‚4.230533777034446 /loss/FRt3/assignment_loss‚4.235580192055813 /loss/FRt4/assignment_loss‚4.605956085990457 .loss/FRt2/detection_loss‚2.618914145006709 1loss/SRqqt4/detection_loss‚2.2353732655445735 epoch‚3 /loss/FBt1/detection_loss‚3.1591909304261208 /loss/FBt4/detection_loss‚3.2877504891819425 trainer/global_step‚2049 _runtime‚1700.680280803 0loss/FRt1/assignment_loss‚3.4719015759426157 .loss/FRt4/detection_loss‚2.517671910804861 1loss/SRqqt3/detection_loss‚1.4150126394815743 /loss/FBt3/detection_loss‚2.3308470903994127 % _timestamp‚1.7883949277006633e+09 _step‚166 .loss/FRt1/detection_loss‚2.353811801298634 2loss/SRqqt4/assignment_loss‚2.4582264605495663 0loss/FBt1/assignment_loss‚0.4489427070261627 1loss/SRqqt1/detection_loss‚1.7368262775005803 /loss/FBt2/detection_loss‚1.0702564530061551 0loss/FBt2/assignment_loss‚0.2779080953348378 -loss/FRt3/detection_loss‚2.20647900950077 0loss/FBt4/assignment_loss‚0.5449609215417391 2loss/SRqqt1/assignment_loss‚1.8934053466433571 2loss/SRqqt2/assignment_loss‚2.6865106503168743 1loss/FBt3/assignment_loss‚0.43017609900871995 0loss/SRqqt2/detection_loss‚3.373492936293284¦0ï‡÷’Ö ‚(:‡ ¹ûâÔÑÖú  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.331' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.8662 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0wD/$s× ‚(:è ¹ûâÔ¤ÿµ$ memory_percent‚86.93380355834961' proc.memory.availableMB‚ 2140.765625 cpu‚24.62524088324556 disk.rbd1.in‚0 disk.rbd1.out‚ 8.349609375 disk.rbd0.out‚ 0.2041015625 disk.nvme0n1.in‚ 338.32421875 disk.sda1.in‚0 disk.drbd1716.in‚0" proc.memory.rssMB‚ 2819.7421875 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda2.out‚0 network.recv‚619431 disk.drbd2266.out‚0 disk.drbd2265.in‚0 disk.rbd0.in‚ 0.0078125) proc.memory.percent‚17.21034049987793 disk.dm-0.in‚ 338.32421875# disk.dm-0.out‚20532.46923828125 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.sda1.out‚0 network.sent‚ 3.882581e+06 disk.sda.in‚0 disk.sda.out‚ 1741.26171875 disk.drbd2266.in‚0 disk.drbd1716.out‚0" disk.nvme0c0n1.in‚ 338.32421875( disk.nvme0c0n1.out‚20532.46923828125* disk./.usagePercent‚16.319941667966503% disk./.usageGB‚243.17892456054688& disk.nvme0n1.out‚20532.46923828125 disk.dm-1.in‚0 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.dm-1.out‚ 1741.26171875 disk.sda3.out‚ 1741.26171875Ù“êÍ’Ø ‚(:‡ Èûâԕÿ  gpu.0.gpu‚21.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚128.642' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.7284 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ú×åërÙ ‚(:ç ÈûâÔ»•À network.recv‚625623 disk.rbd1.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚ 0.2041015625 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.drbd2266.out‚0& disk.nvme0n1.out‚20533.14013671875 disk.sda.out‚ 1760.3828125$ memory_percent‚86.93485260009766 disk.sda1.out‚0 disk.sda3.in‚0& proc.memory.availableMB‚ 2140.59375 cpu‚24.633911015560678% disk./.usageGB‚243.17894744873047 disk.sda3.out‚ 1760.3828125" disk.nvme0c0n1.in‚ 338.32421875( disk.nvme0c0n1.out‚20533.14013671875 disk.dm-0.in‚ 338.32421875 disk.drbd1716.out‚0 disk.rbd1.out‚ 8.3818359375 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 1760.3828125# disk.dm-0.out‚20533.14013671875 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.rbd0.in‚ 0.0078125 proc.cpu.threads‚17* disk./.usagePercent‚16.319943204011707 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd2266.in‚0 network.sent‚ 3.970289e+06 disk.drbd1716.in‚0# proc.memory.rssMB‚ 2819.74609375) proc.memory.percent‚17.21036434173584~äöÌ2Ú - _wandbruntime‚1738  _runtime‚17382VMÇÛ Á % _timestamp‚1.7883949654556937e+09 _runtime‚1738.435477509 _step‚167 & lr-AdamW/pg1‚9.926364012058477e-05 & lr-AdamW/pg2‚9.926364012058477e-05 trainer/global_step‚2099®¡§ÌÜ Æ trainer/global_step‚2099 % _timestamp‚1.7883949654556937e+09 _runtime‚1738.435477509 _step‚167 & lr-AdamW/pg1‚9.926364012058477e-05 & lr-AdamW/pg2‚9.926364012058477e-05§…%œÖ2Ý - _wandbruntime‚1738  _runtime‚1738þ½›ÞÕ'Þ ÏO ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1537437274624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17657102 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10220386 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚112279296817152 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚112279279777792 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚117205166462464 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚115667729187840 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1537601160704 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 % _timestamp‚1.7883949654567852e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚117205347387904 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10219923 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1529317741782 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17657099 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29319716 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚38632425 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38612899 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚2099 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚118610747188736 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1603558659584 _runtime‚ 1738.43686062 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29319714 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29319714 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27877022 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236407808 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚115667729187840 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 <[§ ß ÔO M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚67952141 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29319716 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚38632425 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38612899 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚2099 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚118610747188736 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1603558659584 _runtime‚ 1738.43686062 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29319714 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29319714 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚27877022 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 236407808 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚115667729187840 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1537437274624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17657102 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10220386 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚112279296817152 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚112279279777792 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚117205166462464 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚115667729187840 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1537601160704 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 % _timestamp‚1.7883949654567852e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚117205347387904 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10219923 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1529317741782 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17657099 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚113861399314566 trainer/global_step‚2099 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚113861222202682 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚67964101 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1530184773766 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1530024701242 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38650443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚117060953950208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1604515804160 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚117259448070144 % _timestamp‚1.7883949661380432e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚489 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211490816 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29333212 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29333210 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10224602 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38630891 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10225088 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1604304313344 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1538148021248 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚117259267131392 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚486 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚115721119110144 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17665158 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚67983655 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27889760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚118665494658048 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚115721136149504 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚115721136149504 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1538311920640 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1538148021248 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚118665258263552 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29333210 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚67964101 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚67983655 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29333212 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚117060978853888 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38630891 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _step‚169 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1538311920640 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27890249 _runtime‚1739.118395972 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 PÏgå&â ßM REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17665158 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚67983655 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27889760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚118665494658048 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚115721136149504 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚115721136149504 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1538311920640 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1538148021248 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚118665258263552 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29333210 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚67964101 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚67983655 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29333212 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚117060978853888 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38630891 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 _step‚169 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1538311920640 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27890249 _runtime‚1739.118395972 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚113861399314566 trainer/global_step‚2099 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚113861222202682 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚67964101 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1530184773766 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1530024701242 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38650443 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚117060953950208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1604515804160 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚117259448070144 % _timestamp‚1.7883949661380432e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚489 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211490816 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29333212 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29333210 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10224602 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38630891 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10225088 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1604304313344 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1538148021248 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚117259267131392 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚486 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚115721119110144 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568©eϘ‚2ã - _wandbruntime‚1739  _runtime‚1739~Ú‹Vä Ð .loss/FRt1/assignment_loss‚4.03416459517436 0loss/FBt1/assignment_loss‚0.7975571591175442 0loss/FBt4/assignment_loss‚0.3628233355986595 /loss/FBt3/detection_loss‚3.0726420370241008 trainer/global_step‚2099 0loss/FRt2/assignment_loss‚3.5554938524268396 0loss/FRt3/assignment_loss‚4.7650353410575965 0loss/FRt4/assignment_loss‚4.2133217556723235 0loss/FBt3/assignment_loss‚0.5443832015077229 0loss/SRqqt1/detection_loss‚1.691690131334971 _runtime‚1739.119859305 2loss/SRqqt1/assignment_loss‚1.7275344917537472 /loss/FBt4/detection_loss‚3.8020187675952912 /loss/FRt3/detection_loss‚2.2545647081695024 2loss/SRqqt3/assignment_loss‚2.0297305920747695 .loss/FRt4/detection_loss‚2.520999455931543 0loss/SRqqt2/detection_loss‚2.854197813404931 0loss/SRqqt4/detection_loss‚2.433768067933336 $ _timestamp‚1.788394966140246e+09 epoch‚3 1loss/SRqqt2/assignment_loss‚2.950271877977583 2loss/SRqqt4/assignment_loss‚2.2411101741504065 0loss/FBt2/assignment_loss‚0.2068668366418996 1loss/SRqqt3/detection_loss‚1.7992274077954116 /loss/FBt1/detection_loss‚3.7068288842837016 /loss/FBt2/detection_loss‚1.1395189539944897 %loss/total_loss‚57.20110864922025 /loss/FRt1/detection_loss‚2.2865546890744217 _step‚170 .loss/FRt2/detection_loss‚2.210804519525101k—[å Õ .loss/FRt2/detection_loss‚2.210804519525101 .loss/FRt1/assignment_loss‚4.03416459517436 0loss/FBt1/assignment_loss‚0.7975571591175442 0loss/FBt4/assignment_loss‚0.3628233355986595 /loss/FBt3/detection_loss‚3.0726420370241008 trainer/global_step‚2099 0loss/FRt2/assignment_loss‚3.5554938524268396 0loss/FRt3/assignment_loss‚4.7650353410575965 0loss/FRt4/assignment_loss‚4.2133217556723235 0loss/FBt3/assignment_loss‚0.5443832015077229 0loss/SRqqt1/detection_loss‚1.691690131334971 _runtime‚1739.119859305 2loss/SRqqt1/assignment_loss‚1.7275344917537472 /loss/FBt4/detection_loss‚3.8020187675952912 /loss/FRt3/detection_loss‚2.2545647081695024 2loss/SRqqt3/assignment_loss‚2.0297305920747695 .loss/FRt4/detection_loss‚2.520999455931543 0loss/SRqqt2/detection_loss‚2.854197813404931 0loss/SRqqt4/detection_loss‚2.433768067933336 $ _timestamp‚1.788394966140246e+09 epoch‚3 1loss/SRqqt2/assignment_loss‚2.950271877977583 2loss/SRqqt4/assignment_loss‚2.2411101741504065 0loss/FBt2/assignment_loss‚0.2068668366418996 1loss/SRqqt3/detection_loss‚1.7992274077954116 /loss/FBt1/detection_loss‚3.7068288842837016 /loss/FBt2/detection_loss‚1.1395189539944897 %loss/total_loss‚57.20110864922025 /loss/FRt1/detection_loss‚2.2865546890744217 _step‚170ªÇåÈ{’æ ‚(:‡ ×ûâÔæ×ß  gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚171.841' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.3682 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Q¬»wç ‚(:ì ×ûâÔ¾ªÜ  disk.sda.in‚0 disk.dm-1.out‚ 1779.92578125 disk.sda2.in‚0) proc.memory.availableMB‚ 2140.72265625" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875 disk.drbd1716.in‚0 disk.rbd0.out‚ 0.2041015625& disk.nvme0n1.out‚20533.32373046875 disk.sda1.in‚0( disk.nvme0c0n1.out‚20533.32373046875 disk.rbd0.in‚ 0.0078125 disk.drbd1716.out‚0 disk.drbd2080.in‚0% disk./.usageGB‚243.17892837524414 disk.sda3.in‚0 disk.rbd1.out‚ 8.3818359375# disk.dm-0.out‚20533.32373046875 disk.sda2.out‚0 network.recv‚629658 disk.sda.out‚ 1779.92578125 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 338.32421875$ memory_percent‚86.93406581878662) proc.memory.percent‚17.21036434173584 disk.drbd2266.out‚0 proc.cpu.threads‚17 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.sda3.out‚ 1779.92578125 disk.rbd1.in‚0 network.sent‚ 3.976457e+06) disk./.usagePercent‚16.31994192397404 disk.sda1.out‚0# proc.memory.rssMB‚ 2819.74609375 cpu‚24.642333187046898å];ð’è ‚(:‡ æûâÔÑñÁ  gpu.0.gpu‚29.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.023' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.2046 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0kqKué ‚(:æ æûâÔƒ¶™) proc.memory.percent‚17.21036434173584 disk.dm-1.out‚ 1800.015625 disk.sda1.out‚0g×Cü0 network.sent‚ 4.063004e+06 network.recv‚635189" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875 disk.sda2.in‚0 disk.rbd0.out‚ 0.2041015625% disk./.usageGB‚243.17853164672852# disk.dm-0.out‚20533.76806640625 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.sda.out‚ 1800.015625 disk.drbd2265.in‚0$ memory_percent‚86.93404197692871# proc.memory.rssMB‚ 2819.74609375 disk.drbd1716.out‚0* disk./.usagePercent‚16.319915299190477 disk.drbd2266.out‚0 disk.sda1.in‚0 disk.dm-1.in‚0 disk.rbd1.in‚0( proc.memory.availableMB‚ 2140.7265625 cpu‚24.650851747990608 proc.cpu.threads‚17 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.drbd1716.in‚0( disk.nvme0c0n1.out‚20533.76806640625 disk.sda3.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚ 0.0078125& disk.nvme0n1.out‚20533.76806640625 disk.sda3.out‚ 1800.015625 disk.rbd1.out‚ 8.7099609375*µ.õê ‚(:’ õûâÔ›«ã  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚166.668' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.333600000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0º Q?rë ‚(:ç õûâÔ⳶ network.recv‚638834 disk.drbd2265.in‚0 disk.sda1.out‚0( proc.memory.availableMB‚ 2140.6953125# proc.memory.rssMB‚ 2819.74609375 disk.sda3.out‚ 1818.5859375 disk.rbd1.in‚0 disk.sda2.in‚0 disk.rbd0.out‚ 0.2041015625 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda.in‚0 disk.sda.out‚ 1818.5859375 disk.drbd2266.in‚0 proc.cpu.threads‚17* disk./.usagePercent‚16.319949604200062 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.dm-1.out‚ 1818.5859375 network.sent‚ 4.068454e+06 disk.sda3.in‚0( disk.nvme0c0n1.out‚20533.97119140625 disk.drbd2266.out‚0 disk.rbd1.out‚ 8.7099609375 disk.dm-0.in‚ 338.32421875# disk.dm-0.out‚20533.97119140625 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875$ disk./.usageGB‚243.1790428161621" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.out‚0& disk.nvme0n1.out‚20533.97119140625 disk.sda1.in‚0) proc.memory.percent‚17.21036434173584 cpu‚24.65955558654783 disk.dm-1.in‚0$ memory_percent‚86.93423271179199_ÄÀ±2ì - _wandbruntime‚1777  _runtime‚1777ýR²Çí Á & lr-AdamW/pg1‚9.792508536455075e-05 & lr-AdamW/pg2‚9.792508536455075e-05 trainer/global_step‚2149 % _timestamp‚1.7883950041418533e+09 _runtime‚1777.121648695 _step‚171e—(žÌî Æ _runtime‚1777.121648695 _step‚171 & lr-AdamW/pg1‚9.792508536455075e-05 & lr-AdamW/pg2‚9.792508536455075e-05 trainer/global_step‚2149 % _timestamp‚1.7883950041418533e+09«+³Yû2ï - _wandbruntime‚1777  _runtime‚1777˜)/ìÖ'ð ÐO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29994516 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1573166320128 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚39535377 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18060052 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚457 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚114875165960192 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚119910408662528 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚118337225303040 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18060049 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚39535377 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚118337242342400 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 248990720 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224087040 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _step‚172 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1573002434048 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 trainer/global_step‚2149 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1573002434048 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28512710 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚69510365 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29994514 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚454 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1640966431232 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚119910227737088 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29994514 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚119707068468224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1564692886790 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚116440035958658 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29994516 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚119910408662528 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚118337242342400 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39515851 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚121347785908736 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚114875182999552 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1573166320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10453115 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚121348034899456 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1640742344192 % _timestamp‚1.7883950041428704e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚119910227737088 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚118337225303040 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10452661 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚69529893 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚119707043564544 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676y‡LÛ'ñ ÕO PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚39535377 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚118337242342400 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 248990720 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 224087040 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _step‚172 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1573002434048 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 trainer/global_step‚2149 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1573002434048 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28512710 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚69510365 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29994514 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚454 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1640966431232 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚119910227737088 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29994514 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚119707068468224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1564692886790 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚116440035958658 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29994516 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚119910408662528 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚118337242342400 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39515851 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚121347785908736 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚114875182999552 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1573166320128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10453115 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚121348034899456 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1640742344192 % _timestamp‚1.7883950041428704e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚119910227737088 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚118337225303040 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10452661 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚69529893 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚119707043564544 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29994516 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1573166320128 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚39535377 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18060052 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚457 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚114875165960192 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚119910408662528 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚118337225303040 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18060049 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765¬Ô›ª2ò - _wandbruntime‚1777  _runtime‚1777»ûº[Î#ó ÚM F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18068108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚39553395 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 % _timestamp‚1.7883950048234406e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚69561407 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚119761070297088 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚116492660641974 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114927083683840 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10457684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1641701270528 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚69561407 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _step‚173 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚118390615225344 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚439 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚119761045393408 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1565559918774 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28525356 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18068111 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚121402533075968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚119964509344768 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚118390632264704 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚436 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1565399846250 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚69541853 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1573877080064 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238491648 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 trainer/global_step‚2149 _runtime‚1777.803598187 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚118390615225344 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚39533843 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1641487682560 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1573713180672 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1573877080064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30008012 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚119964509344768 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.pea e“Lk‚2513 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚116492483530090 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚119964328406016 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39553395 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚114927100723200 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207VŸœ8å&ô ßM WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚119964509344768 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚116492483530090 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚119964328406016 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39553395 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚114927100723200 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18068108 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚39553395 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 % _timestamp‚1.7883950048234406e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚69561407 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚119761070297088 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚116492660641974 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114927083683840 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10457684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1641701270528 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚69561407 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _step‚173 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚118390615225344 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚439 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚119761045393408 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1565559918774 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28525356 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18068111 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚121402533075968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚119964509344768 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚118390632264704 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚436 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1565399846250 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚69541853 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1573877080064 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 238491648 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 trainer/global_step‚2149 _runtime‚1777.803598187 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚118390615225344 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚39533843 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1641487682560 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1573713180672 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1573877080064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30008012 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392­/ZÀ2õ - _wandbruntime‚1777  _runtime‚1777>OrûWö Ñ /loss/FRt3/assignment_loss‚4.098732144936271 1loss/SRqqt1/detection_loss‚1.6313282031703877 1loss/SRqqt3/detection_loss‚1.5286020936492162 /loss/FBt2/detection_loss‚1.2032612128119455 _runtime‚1777.804963554 2loss/SRqqt3/assignment_loss‚1.8281439897628167 /loss/FBt2/assignment_loss‚0.284398979737423 /loss/FRt1/detection_loss‚2.3217611932060094 /loss/FRt4/detection_loss‚2.5589720853945104 1loss/SRqqt2/detection_loss‚2.6425210389685123 1loss/SRqqt4/detection_loss‚2.3401295449360306 .loss/FRt2/detection_loss‚2.399661841669253 0loss/FRt1/assignment_loss‚3.6713581154647383 .loss/FBt1/detection_loss‚3.191018473534357 .loss/FBt4/detection_loss‚3.536382704973221 1loss/SRqqt1/assignment_loss‚1.757124413660279 1loss/SRqqt4/assignment_loss‚2.107997063771788 0loss/FBt1/assignment_loss‚0.6674500057538569 /loss/FBt3/detection_loss‚2.5719128888229226 epoch‚3 % _timestamp‚1.7883950048250387e+09 /loss/FRt4/assignment_loss‚4.224751087223611 2loss/SRqqt2/assignment_loss‚2.8052992123238583 /loss/FRt3/detection_loss‚2.1941690487058265 %loss/total_loss‚54.02351877697691 0loss/FRt2/assignment_loss‚3.3279856954302107 0loss/FBt3/assignment_loss‚0.6550974443778551 0loss/FBt4/assignment_loss‚0.4754602946920083 trainer/global_step‚2149 _step‚174f)$Ä\÷ Ö 0loss/FRt2/assignment_loss‚3.3279856954302107 0loss/FBt3/assignment_loss‚0.6550974443778551 0loss/FBt4/assignment_loss‚0.4754602946920083 trainer/global_step‚2149 _step‚174 /loss/FRt3/assignment_loss‚4.098732144936271 1loss/SRqqt1/detection_loss‚1.6313282031703877 1loss/SRqqt3/detection_loss‚1.5286020936492162 /loss/FBt2/detection_loss‚1.2032612128119455 _runtime‚1777.804963554 2loss/SRqqt3/assignment_loss‚1.8281439897628167 /loss/FBt2/assignment_loss‚0.284398979737423 /loss/FRt1/detection_loss‚2.3217611932060094 /loss/FRt4/detection_loss‚2.5589720853945104 1loss/SRqqt2/detection_loss‚2.6425210389685123 1loss/SRqqt4/detection_loss‚2.3401295449360306 .loss/FRt2/detection_loss‚2.399661841669253 0loss/FRt1/assignment_loss‚3.6713581154647383 .loss/FBt1/detection_loss‚3.191018473534357 .loss/FBt4/detection_loss‚3.536382704973221 1loss/SRqqt1/assignment_loss‚1.757124413660279 1loss/SRqqt4/assignment_loss‚2.107997063771788 0loss/FBt1/assignment_loss‚0.6674500057538569 /loss/FBt3/detection_loss‚2.5719128888229226 epoch‚3 % _timestamp‚1.7883950048250387e+09 /loss/FRt4/assignment_loss‚4.224751087223611 2loss/SRqqt2/assignment_loss‚2.8052992123238583 /loss/FRt3/detection_loss‚2.1941690487058265 %loss/total_loss‚54.02351877697691®ËV¡¥’ø ‚(:‡ „üâÔÓ–ý  gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚172.041' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4082 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚09½(eù ‚(:Ú „üâÔ¿†ž disk.sda3.in‚0 disk.rbd1.in‚0 network.sent‚ 4.074999e+06 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.drbd2080.in‚0* disk./.usagePercent‚16.319953444313075 disk.dm-0.in‚ 338.32421875 disk.nvme0n1.in‚ 338.32421875 proc.memory.rssMB‚2819.75 cpu‚24.667644040026012 network.recv‚643695" disk.nvme0c0n1.in‚ 338.32421875 disk.rbd0.in‚ 0.0078125 disk.rbd0.out‚ 0.2041015625 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 8.7421875) proc.memory.percent‚17.21038818359375 proc.cpu.threads‚17 disk.drbd2266.in‚0$ memory_percent‚86.93602085113525) proc.memory.availableMB‚ 2140.40234375$ disk.nvme0n1.out‚20534.529296875 disk.sda.out‚ 1836.8828125 disk.dm-1.in‚0 disk.dm-1.out‚ 1836.8828125 disk.drbd2080.out‚0& disk.nvme0c0n1.out‚20534.529296875 disk.sda2.out‚0 disk.drbd1716.out‚0$ disk./.usageGB‚243.1791000366211 disk.sda3.out‚ 1836.8828125! disk.dm-0.out‚20534.529296875 disk.drbd1716.in‚0 disk.sda.in‚0 disk.drbd2266.out‚0I·h’ú ‚(:‡ “üâÔ®Øú  gpu.0.gpu‚42.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚177.174' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.4348 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ê®_cû ‚(:Ø “üâÔ¬ì network.recv‚649810 disk.drbd2080.in‚0 disk.rbd0.out‚ 0.2041015625 disk.rbd1.in‚0 disk.rbd1.out‚ 8.7421875 disk.sda.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1854.6640625$ memory_percent‚86.93609237670898" disk.nvme0c0n1.in‚ 338.32421875! disk.dm-0.out‚20534.931640625 disk.nvme0n1.in‚ 338.32421875 disk.drbd2265.in‚0* disk./.usagePercent‚16.319949604200062 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0& disk.nvme0c0n1.out‚20534.931640625 disk.sda2.out‚0 disk.drbd2266.out‚0) proc.memory.percent‚17.21038818359375 proc.cpu.threads‚17 disk.sda3.out‚ 1854.6640625 disk.drbd1716.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 cpu‚24.675528156791987 disk.dm-0.in‚ 338.32421875 disk.sda2.in‚0 disk.drbd2266.in‚0 proc.memory.rssMB‚2819.75 network.sent‚ 4.162472e+06$ disk./.usageGB‚243.1790428161621 disk.rbd0.in‚ 0.0078125$ disk.nvme0n1.out‚20534.931640625 disk.sda.out‚ 1854.6640625 disk.sda1.in‚0' proc.memory.availableMB‚ 2140.390625—¿cBü ‚(:‚ ¢üâÔבþ  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚175.0' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.0 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ð‹ùhý ‚(:Ý ¢üâÔ¸þ) proc.memory.percent‚17.21038818359375 network.sent‚ 4.167911e+06* disk./.usagePercent‚16.319949604200062 disk.rbd0.in‚ 0.0078125 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 1873.80859375$ memory_percent‚86.93456649780273 disk.sda2.in‚0$ disk./.usageGB‚243.1790428161621 disk.sda3.out‚ 1873.80859375" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875" disk.dm-0.out‚20535.3369140625 disk.nvme0n1.in‚ 338.32421875 disk.sda.out‚ 1873.80859375 disk.drbd1716.in‚0 disk.sda.in‚0 proc.memory.rssMB‚2819.75 disk.rbd1.in‚0 disk.rbd1.out‚ 8.8984375' proc.memory.availableMB‚ 2140.640625 cpu‚24.68323204812296 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0% disk.nvme0n1.out‚20535.3369140625 disk.drbd2265.out‚0 disk.sda1.out‚0 proc.cpu.threads‚17 disk.sda2.out‚0' disk.nvme0c0n1.out‚20535.3369140625 disk.rbd0.out‚ 0.2041015625 disk.drbd2266.out‚0 disk.sda1.in‚0 network.recv‚653727 disk.drbd2080.out‚0ت߿2þ - _wandbruntime‚1815  _runtime‚1815Ÿ€ûöÇÿ Á % _timestamp‚1.7883950428225431e+09 _runtime‚1815.802344502 _step‚175 & lr-AdamW/pg1‚9.655501732901089e-05 & lr-AdamW/pg2‚9.655501732901089e-05 trainer/global_step‚2199x|5ÈÌ€Æ & lr-AdamW/pg1‚9.655501732901089e-05 & lr-AdamW/pg2‚9.655501732901089e-05 trainer/global_step‚2199 % _timestamp‚1.7883950428225431e+09 _runtime‚1815.802344502 _step‚175¯¨*]?2- _wandbruntime‚1815  _runtime‚1815ŸoÌåÕ'‚ÏO UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚122615289011712 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1608731479552 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30669314 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10686469 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1608567593472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚122615289011712 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚122407135007744 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10686936 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40438329 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40438329 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1678119940608 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1677904242176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚121006738457600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚122615469937152 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚29149938 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809ñµøƒÔO ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1608731479552 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30669314 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10686469 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1608567593472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚122615289011712 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚122407135007744 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10686936 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40438329 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40438329 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1678119940608 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1677904242176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚121006738457600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚122615469937152 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚29149938 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚122615289011712 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360°D¯VÖ2„- _wandbruntime‚1816  _runtime‚1816ƒ|¡à&…ÚM PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71139159 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211490816 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚71139159 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30682812 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29162166 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚454 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚117522969866240 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚30682810 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚457 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 % _timestamp‚1.7883950435204992e+09 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚121060128379904 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚121060111340544 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40436795 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18471061 _step‚177 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚71119605 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40456347 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18471058 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1609278340096 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚122669389680640 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚119123921969382 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚122461136836608 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚122669570619392 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1678862134272 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚117522986905600 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1600935063782 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40436795 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚121060128379904 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10691562 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1678650643456 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71119605 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚124140023874560 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 trainer/global_step‚2199 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1609442239488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207ü€%zå&†ßM HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚119123921969382 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚122461136836608 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚122669570619392 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1678862134272 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚117522986905600 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1600935063782 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40436795 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚121060128379904 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10691562 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1678650643456 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71119605 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚124140023874560 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 trainer/global_step‚2199 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1609442239488 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71139159 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 211490816 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚71139159 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30682812 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29162166 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚454 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚117522969866240 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚30682810 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚457 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 % _timestamp‚1.7883950435204992e+09 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚121060128379904 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚121060111340544 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40436795 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18471061 _step‚177 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚71119605 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40456347 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18471058 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1609278340096 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚122669389680640 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623±0ØÏœ2‡- _wandbruntime‚1816  _runtime‚1816m”ÒàWˆÑ 2loss/SRqqt1/assignment_loss‚1.9849724874776953 1loss/SRqqt1/detection_loss‚1.4753010010505034 0loss/FRt3/assignment_loss‚3.6985525941848754 /loss/FRt4/assignment_loss‚4.075718855857849 1loss/SRqqt3/detection_loss‚1.7953265259663265 /loss/FBt2/detection_loss‚0.9080897900646432 _runtime‚1816.502122934 .loss/FRt2/detection_loss‚2.295160800218582 1loss/SRqqt4/detection_loss‚2.5412524485588075 -loss/FRt1/detection_loss‚2.19622223621065 0loss/FBt1/assignment_loss‚0.4583113021575488 1loss/FBt2/assignment_loss‚0.23475909501211156 .loss/FBt4/detection_loss‚3.928519546985626 0loss/FRt2/assignment_loss‚3.4310781570929514 /loss/FBt1/detection_loss‚3.6804468723443837 .loss/FBt3/detection_loss‚2.755049803772488 epoch‚3 % _timestamp‚1.7883950435224621e+09 trainer/global_step‚2199 0loss/FBt4/assignment_loss‚0.4536655056686464 0loss/FRt1/assignment_loss‚3.8607921281246225 1loss/SRqqt2/assignment_loss‚3.004607461116932 /loss/FRt4/detection_loss‚2.6021274011582136 1loss/SRqqt4/assignment_loss‚2.048285334507624 .loss/FRt3/detection_loss‚2.171212865114212 0loss/SRqqt2/detection_loss‚3.334825277328491 2loss/SRqqt3/assignment_loss‚1.6720782470490252 1loss/FBt3/assignment_loss‚0.46686381618899364 %loss/total_loss‚55.07321955321181 _step‚178×<È\‰Ö 1loss/SRqqt4/assignment_loss‚2.048285334507624 .loss/FRt3/detection_loss‚2.171212865114212 0loss/SRqqt2/detection_loss‚3.334825277328491 2loss/SRqqt3/assignment_loss‚1.6720782470490252 1loss/FBt3/assignment_loss‚0.46686381618899364 %loss/total_loss‚55.07321955321181 _step‚178 2loss/SRqqt1/assignment_loss‚1.9849724874776953 1loss/SRqqt1/detection_loss‚1.4753010010505034 0loss/FRt3/assignment_loss‚3.6985525941848754 /loss/FRt4/assignment_loss‚4.075718855857849 1loss/SRqqt3/detection_loss‚1.7953265259663265 /loss/FBt2/detection_loss‚0.9080897900646432 _runtime‚1816.502122934 .loss/FRt2/detection_loss‚2.295160800218582 1loss/SRqqt4/detection_loss‚2.5412524485588075 -loss/FRt1/detection_loss‚2.19622223621065 0loss/FBt1/assignment_loss‚0.4583113021575488 1loss/FBt2/assignment_loss‚0.23475909501211156 .loss/FBt4/detection_loss‚3.928519546985626 0loss/FRt2/assignment_loss‚3.4310781570929514 /loss/FBt1/detection_loss‚3.6804468723443837 .loss/FBt3/detection_loss‚2.755049803772488 epoch‚3 % _timestamp‚1.7883950435224621e+09 trainer/global_step‚2199 0loss/FBt4/assignment_loss‚0.4536655056686464 0loss/FRt1/assignment_loss‚3.8607921281246225 1loss/SRqqt2/assignment_loss‚3.004607461116932 /loss/FRt4/detection_loss‚2.6021274011582136²ž:LjŠ‚(:’ ±üâÔ¥û£  gpu.0.gpu‚37.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚181.517' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚36.303399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ä½”g‹‚(:Ü ±üâÔÈ×  disk.sda3.in‚0" disk.nvme0c0n1.in‚ 338.32421875 cpu‚24.691059534947943 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.drbd2265.in‚0 disk.dm-0.in‚ 338.32421875 disk.drbd1716.in‚0% disk.nvme0n1.out‚20535.8095703125 disk.dm-1.in‚0) proc.memory.percent‚17.21038818359375 disk.drbd2266.out‚0' proc.memory.availableMB‚ 2140.390625 proc.memory.rssMB‚2819.75 disk.drbd1716.out‚0 disk.sda.out‚ 1894.3359375 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1894.3359375 network.recv‚657001 disk.drbd2080.out‚0 disk.rbd0.in‚ 0.0078125 disk.sda.in‚0' disk.nvme0c0n1.out‚20535.8095703125* disk./.usagePercent‚16.320018470226774% disk./.usageGB‚243.18006896972656 disk.sda2.in‚0 disk.sda3.out‚ 1894.3359375" disk.dm-0.out‚20535.8095703125$ memory_percent‚86.93609237670898 network.sent‚ 4.173172e+06 disk.drbd2265.out‚0 disk.rbd1.out‚ 8.8984375 disk.rbd0.out‚ 0.2041015625 disk.nvme0n1.in‚ 338.32421875 disk.sda1.in‚0u‰`ŽjŒ‚(:ß ÀüâÔ„àã  disk.dm-1.in‚0$ disk./.usageGB‚243.1807632446289 disk.rbd1.out‚ 8.9306640625 disk.drbd1716.out‚0 disk.rbd0.out‚ 0.2041015625 disk.sda.in‚0 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1915.5390625' disk.nvme0c0n1.out‚20537.9208984375" disk.dm-0.out‚20537.9208984375( proc.memory.availableMB‚ 2140.6328125) proc.memory.percent‚17.21038818359375 disk.drbd2265.in‚0 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.sda2.in‚0 disk.drbd2080.out‚0$ memory_percent‚86.93461418151855 proc.cpu.threads‚17 disk.sda3.in‚0 disk.sda3.out‚ 1915.5390625 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 338.32421875 disk.sda1.in‚0 disk.drbd2266.out‚0 proc.memory.rssMB‚2819.75 network.recv‚663683 network.sent‚ 4.260858e+06 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.sda.out‚ 1915.5390625 cpu‚24.698704853990844* disk./.usagePercent‚16.320065063598005 disk.dm-0.in‚ 338.32421875 disk.rbd0.in‚ 0.0078125% disk.nvme0n1.out‚20537.9208984375 disk.drbd2265.out‚0¥u}m’‚(:‡ ÀüâԆɣ gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.023' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.6046 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0"L%à2Ž- _wandbruntime‚1854  _runtime‚1854(UE`ÇÁ _step‚179 & lr-AdamW/pg1‚9.515457992637785e-05 & lr-AdamW/pg2‚9.515457992637785e-05 trainer/global_step‚2249 % _timestamp‚1.7883950816148493e+09 _runtime‚1854.594673745>E*ŠÌÆ _runtime‚1854.594673745 _step‚179 & lr-AdamW/pg1‚9.515457992637785e-05 & lr-AdamW/pg2‚9.515457992637785e-05 trainer/global_step‚2249 % _timestamp‚1.7883950816148493e+09³9£Õ­‘- _wandbruntime‚1ØmÓ854  _runtime‚1854 ½_UÖ'’ÐO NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31344114 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10920348 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚123676217533440 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚125320350286336 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 209406976 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1635443176806 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚72685397 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚125107226450944 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1644296638976 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚121702558613474 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31344116 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚121702381501798 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31344116 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚125107251354624 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18865949 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚120066955364352 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31344114 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚126822521300480 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1635603249122 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚125320531211776 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1715060538880 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚125320531211776 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚123676234572800 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1715269945856 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18865952 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1644296638976 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚41341281 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚125320350286336 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41341281 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚125320350286336 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41341281 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31344116 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚121702381501798 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31344116 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚125107251354624 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18865949 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚120066955364352 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31344114 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚126822521300480 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1635603249122 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚125320531211776 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1715060538880 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚125320531211776 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚123676234572800 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1715269945856 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18865952 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496´ö®Ü2”- _wandbruntime‚1855  _runtime‚1855zQÔß&•ÙM ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚123729607455744 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72716911 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31357612 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31357610 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18874008 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 trainer/global_step‚2249 $ _timestamp‚1.788395082308504e+09 _step‚181 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29798885 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚125374631894016 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41339747 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚123729624495104 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1644843499520 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1645007398912 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚125161228279808 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚123729624495104 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚126877040448512 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1644843499520 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1715812168704 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚120118856048640 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72716911 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31357612 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 223811584 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚126877264260096 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚125161253183488 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚121755183296790 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚41359299 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚436 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 _runtime‚1855.288812435 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1716011076608 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚121755006184906 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚125374631894016 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72697357 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41339747 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41359299 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1636310208790 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416V,Æþ–ÞM =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1644843499520 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 177111884 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1715812168704 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚120118856048640 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72716911 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31357612 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 223811584 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚126877264260096 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚125161253183488 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚121755183296790 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMo;úm4ænitor.on_train_batch_end/allocation.small_pool.allocated‚41359299 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚436 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 _runtime‚1855.288812435 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1716011076608 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚121755006184906 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚125374631894016 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72697357 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41339747 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41359299 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1636310208790 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚123729607455744 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72716911 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31357612 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31357610 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18874008 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 trainer/global_step‚2249 $ _timestamp‚1.788395082308504e+09 _step‚181 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29798885 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚125374631894016 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41339747 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚123729624495104 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1644843499520 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1645007398912 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚125161228279808 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚123729624495104 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚126877040448512µ‚ÙED2—- _wandbruntime‚1855  _runtime‚1855$ôðT˜Î /loss/FRt2/assignment_loss‚3.854056585002953 2loss/SRqqt3/assignment_loss‚1.8707120706365532 0loss/FBt1/assignment_loss‚0.4793846661234152 .loss/FRt4/detection_loss‚2.793340233595748 0loss/SRqqt4/detection_loss‚2.457594739894072 /loss/FBt2/detection_loss‚1.0636199167362181 trainer/global_step‚2249 % _timestamp‚1.7883950823102937e+09 2loss/SRqqt1/assignment_loss‚1.8843946454733829 0loss/FRt3/assignment_loss‚3.8692030199738436 2loss/SRqqt2/assignment_loss‚2.9531702384716128 0loss/FBt3/assignment_loss‚0.4060880475115954 .loss/FRt1/detection_loss‚2.380575039170005 .loss/FRt2/detection_loss‚2.466461495824263 1loss/SRqqt3/detection_loss‚1.6144176680747777 _step‚182 /loss/FRt1/assignment_loss‚4.019916705109856 2loss/SRqqt4/assignment_loss‚2.2600895747956304 0loss/FBt4/assignment_loss‚0.2211812346180927 .loss/FBt3/detection_loss‚7.940345095263589 $loss/total_loss‚60.0689999910048 epoch‚3 0loss/SRqqt1/detection_loss‚1.691728945739948 .loss/FBt1/detection_loss‚3.045315262518431 /loss/FBt4/detection_loss‚3.4326468772358365 /loss/FRt4/assignment_loss‚4.113772022096734 0loss/SRqqt2/detection_loss‚2.821514466913735 _runtime‚1855.289945664 1loss/FBt2/assignment_loss‚0.23867733616404732 /loss/FRt3/detection_loss‚2.1907941040604615/b3Y™Ó /loss/FRt3/detection_loss‚2.1907941040604615 /loss/FRt2/assignment_loss‚3.854056585002953 2loss/SRqqt3/assignment_loss‚1.8707120706365532 0loss/FBt1/assignment_loss‚0.4793846661234152 .loss/FRt4/detection_loss‚2.793340233595748 0loss/SRqqt4/detection_loss‚2.457594739894072 /loss/FBt2/detection_loss‚1.0636199167362181 trainer/global_step‚2249 % _timestamp‚1.7883950823102937e+09 2loss/SRqqt1/assignment_loss‚1.8843946454733829 0loss/FRt3/assignment_loss‚3.8692030199738436 2loss/SRqqt2/assignment_loss‚2.9531702384716128 0loss/FBt3/assignment_loss‚0.4060880475115954 .loss/FRt1/detection_loss‚2.380575039170005 .loss/FRt2/detection_loss‚2.466461495824263 1loss/SRqqt3/detection_loss‚1.6144176680747777 _step‚182 /loss/FRt1/assignment_loss‚4.019916705109856 2loss/SRqqt4/assignment_loss‚2.2600895747956304 0loss/FBt4/assignment_loss‚0.2211812346180927 .loss/FBt3/detection_loss‚7.940345095263589 $loss/total_loss‚60.0689999910048 epoch‚3 0loss/SRqqt1/detection_loss‚1.691728945739948 .loss/FBt1/detection_loss‚3.045315262518431 /loss/FBt4/detection_loss‚3.4326468772358365 /loss/FRt4/assignment_loss‚4.113772022096734 0loss/SRqqt2/detection_loss‚2.821514466913735 _runtime‚1855.289945664 1loss/FBt2/assignment_loss‚0.23867733616404732¶%™&’š‚(:‡ ÏüâÔ‘ïõ  gpu.0.gpu‚50.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚173.442' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.6884 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0N¢4m›‚(:â ÏüâÔ§¶ disk.drbd2080.out‚0 disk.rbd0.out‚ 0.2041015625 disk.sda3.in‚0 disk.nvme0n1.in‚ 338.32421875 proc.cpu.threads‚17 disk.drbd2266.in‚0 disk.rbd0.in‚ 0.0078125$ memory_percent‚86.93649768829346) proc.memory.availableMB‚ 2140.32421875) disk./.usagePercent‚16.32006122348499 disk.sda3.out‚ 1933.96484375 disk.rbd1.in‚0 disk.drbd1716.out‚0 disk.drbd2265.in‚0 disk.rbd1.out‚ 8.9306640625" disk.nvme0c0n1.in‚ 338.32421875' disk.nvme0c0n1.out‚20538.2021484375% disk.nvme0n1.out‚20538.2021484375 proc.memory.rssMB‚2819.75) proc.memory.percent‚17.21038818359375 network.recv‚667814% disk./.usageGB‚243.18070602416992 disk.drbd2266.out‚0" disk.dm-0.out‚20538.2021484375 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda.in‚0 disk.sda.out‚ 1933.96484375 disk.sda1.in‚0 disk.dm-1.out‚ 1933.96484375 cpu‚24.70610324479193 disk.dm-0.in‚ 338.32421875 disk.drbd2080.in‚0 network.sent‚ 4.267025e+06 disk.drbd2265.out‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0›sk’œ‚(:‡ ÞüâÔ䨣  gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚124.046' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.8092 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚04â;h‚(:Ý ÞüâÔ¿•…  disk.sda2.out‚0 disk.rbd0.out‚ 0.2041015625 disk.nvme0n1.in‚ 338.32421875% disk./.usageGB‚243.18070602416992 disk.dm-0.in‚ 338.32421875 disk.rbd0.in‚ 0.0078125 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 9.1728515625" disk.nvme0c0n1.in‚ 338.32421875$ disk.nvme0n1.out‚20538.521484375$ memory_percent‚86.93647384643555 proc.memory.rssMB‚2819.75) proc.memory.percent‚17.21038818359375 disk.drbd1716.in‚0 disk.drbd1716.out‚0' proc.memory.availableMB‚ 2140.328125 network.recv‚673302 disk.sda.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.drbd2265.in‚0 disk.sda1.out‚0 disk.dm-1.out‚ 1946.62890625& disk.nvme0c0n1.out‚20538.521484375 disk.drbd2080.in‚0 proc.cpu.threads‚17 network.sent‚ 4.35366e+06 disk.sda.out‚ 1946.62890625 disk.dm-1.in‚0 disk.sda3.in‚0! disk.dm-0.out‚20538.521484375 disk.drbd2080.out‚0 cpu‚24.713676995388784) disk./.usagePercent‚16.32006122348499 disk.drbd2266.in‚0 disk.sda3.out‚ 1946.62890625›2êž‚(:’ íüâԹϓ  gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚148.601' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.720200000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ú\3îjŸ‚(:ß íüâÔýóú  disk.rbd0.in‚ 0.0078125 disk.rbd0.out‚ 0.2041015625 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.rbd1.out‚ 9.1728515625( proc.memory.availableMB‚ 2140.2734375 cpu‚24.720963796315033$ disk./.usageGB‚243.1807098388672 disk.sda2.in‚0 disk.sda1.out‚0$ memory_percent‚86.93680763244629 disk.sda2.out‚0 disk.drbd1716.in‚0 proc.memory.rssMB‚2819.75 disk.drbd2080.in‚0 disk.sda.out‚ 1965.87890625 disk.drbd2266.out‚0 network.recv‚676713 disk.sda3.in‚0& disk.nvme0c0n1.out‚20538.705078125) proc.memory.percent‚17.21038818359375 proc.cpu.threads‚17 disk.nvme0n1.in‚ 338.32421875 disk.drbd2266.in‚0 disk.dm-1.out‚ 1965.87890625 disk.sda3.out‚ 1965.87890625 disk.rbd1.in‚0* disk./.usagePercent‚16.320061479492527 disk.dm-0.in‚ 338.32421875! disk.dm-0.out‚20538.705078125$ disk.nvme0n1.out‚20538.705078125 disk.sda.in‚0 disk.sda1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 network.sent‚ 4.359034e+06 disk.drbd1716.out‚0 disk.drbd2265.in‚0 disk.drbd2080.out‚0ž`Ú2 - _wandbruntime‚1893  _runtime‚1893‹hÇ¡Á _step‚183 & lr-AdamW/pg1‚9.372494242539337e-05 & lr-AdamW/pg2‚9.372494242539337e-05 trainer/global_step‚2299 % _timestamp‚1.7883951206333075e+09 _runtime‚1893.613182066c[›*Ì¢Æ trainer/global_step‚2299 % _timestamp‚1.7883951206333075e+09 _runtime‚1893.613182066 _step‚183 & lr-AdamW/pg1‚9.372494242539337e-05 & lr-AdamW/pg2‚9.372494242539337e-05·êwù2£- _wandbruntime‚1893  _runtime‚1893ü»‹Ö'¤ÐO KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32018914 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1670818321814 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1679697912320 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11153438 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30422340 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32018914 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1679861798400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883951206343997e+09 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚129559773112832 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42224707 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚128025592486400 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 _step‚184 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42244233 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1679861798400 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32018916 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚124333819940882 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚128025411560960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11152981 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚126345713648640 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42244233 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1752430315008 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚126345730688000 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚127807342797824 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74243621 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚128025592486400 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚124333642829206 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚127807317894144 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230116352Áþ¥ÕO LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1670978394130 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚128025411560960 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚74263149 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚30421880 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19268902 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19268899 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚122662841546752 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚460 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1679697912320 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚126345713648640 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚126345730688000 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚129559542996480 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32018916 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1752225102336 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 trainer/global_step‚2299 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚126345713648640 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42244233 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1752430315008 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚126345730688000 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚127807342797824 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74243621 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚128025592486400 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚124333642829206 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚127807317894144 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230116352 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32018914 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1670818321814 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1679697912320 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11153438 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30422340 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3201891|xIÝ 4 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1679861798400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883951206343997e+09 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚129559773112832 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42224707 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚128025592486400 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 _step‚184 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42244233 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1679861798400 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32018916 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚124333819940882 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚128025411560960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11152981 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚129614281904128 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1680572558336 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30434532 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚126399120610304 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30434995 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚460 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11157574 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 221976576 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚1894.304962825 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 % _timestamp‚1.7883951213245661e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1680572558336 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32032410 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42262251 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32032412 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42262251 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚126399103570944 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚127861319723008 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74294663 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚74275109 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32032410 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11158034 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚129614528784384 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 246880256 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚127861344626688 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚128079512229888 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42242699 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _step‚185 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74294663 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32032412 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚74275109 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚128079693168640 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736¡C ;å&¨ßM ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32032410 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 180938752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42262251 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32032412 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42262251 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚126399103570944 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚127861319723008 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74294663 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚74275109 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32032410 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11158034 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚129614528784384 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 246880256 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚127861344626688 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚128079512229888 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42242699 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _step‚185 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74294663 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32032412 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚74275109 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚128079693168640 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 422324736 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚124386267512314 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚122714759270400 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚126399103570944 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1752962181120 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚124386444624198 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚129614281904128 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1680572558336 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30434532 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚126399120610304 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30434995 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚460 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11157574 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 221976576 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 _runtime‚1894.304962825 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 % _timestamp‚1.7883951213245661e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1680572558336¹£qÿ‹2©- _wandbruntime‚1894  _runtime‚1894«d7 ZªÔ 2loss/SRqqt2/assignment_loss‚3.0261601752705043 1loss/FBt1/assignment_loss‚0.40353489980837437 1loss/FBt4/assignment_loss‚0.39688614146490137 .loss/FRt3/detection_loss‚2.200047966254794 %loss/total_loss‚53.40635311287819 _step‚186 2loss/SRqqt1/assignment_loss‚1.7847971840073233 2loss/SRqqt4/assignment_loss‚1.9281446452324207 1loss/FBt3/assignment_loss‚0.44917949381860206 .loss/FRt1/detection_loss‚2.262051504519251 epoch‚3 1loss/FBt2/assignment_loss‚0.25443927183026716 .loss/FRt2/detection_loss‚2.220735196556364 1loss/SRqqt2/detection_loss‚2.6619510723484887 0loss/SRqqt3/detection_loss‚1.618550942960515 % _timestamp‚1.7883951213266492e+09 /loss/FRt3/assignment_loss‚3.649341826853545 /loss/FRt4/assignment_loss‚4.497581323410603 2loss/SRqqt3/assignment_loss‚1.9236479681541225 1loss/SRqqt1/detection_loss‚1.6604166747867197 /loss/FBt2/detection_loss‚0.8383001495984311 trainer/global_step‚2299 /loss/FBt4/detection_loss‚3.7449246020544145 /loss/FRt4/detection_loss‚2.5086535196355046 0loss/SRqqt4/detection_loss‚2.310989484955103 .loss/FBt1/detection_loss‚2.940103816298338 /loss/FBt3/detection_loss‚2.5959953665809365 /loss/FRt2/assignment_loss‚3.738631476958593 _runtime‚1894.306297312 0loss/FRt1/assignment_loss‚3.7912884095200785¦ÛJ_«Ù /loss/FRt4/assignment_loss‚4.497581323410603 2loss/SRqqt3/assignment_loss‚1.9236479681541225 1loss/SRqqt1/detection_loss‚1.6604166747867197 /loss/FBt2/detection_loss‚0.8383001495984311 trainer/global_step‚2299 /loss/FBt4/detection_loss‚3.7449246020544145 /loss/FRt4/detection_loss‚2.5086535196355046 0loss/SRqqt4/detection_loss‚2.310989484955103 .loss/FBt1/detection_loss‚2.940103816298338 /loss/FBt3/detection_loss‚2.5959953665809365 /loss/FRt2/assignment_loss‚3.738631476958593 _runtime‚1894.306297312 0loss/FRt1/assignment_loss‚3.7912884095200785 2loss/SRqqt2/assignment_loss‚3.0261601752705043 1loss/FBt1/assignment_loss‚0.40353489980837437 1loss/FBt4/assignment_loss‚0.39688614146490137 .loss/FRt3/detection_loss‚2.200047966254794 %loss/total_loss‚53.40635311287819 _step‚186 2loss/SRqqt1/assignment_loss‚1.7847971840073233 2loss/SRqqt4/assignment_loss‚1.9281446452324207 1loss/FBt3/assignment_loss‚0.44917949381860206 .loss/FRt1/detection_loss‚2.262051504519251 epoch‚3 1loss/FBt2/assignment_loss‚0.25443927183026716 .loss/FRt2/detection_loss‚2.220735196556364 1loss/SRqqt2/detection_loss‚2.6619510723484887 0loss/SRqqt3/detection_loss‚1.618550942960515 % _timestamp‚1.7883951213266492e+09 /loss/FRt3/assignment_loss‚3.649341826853545ºÔÖ×’¬‚(:‡ üüâÔØûü  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚174.222' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.8444 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±Xµh­‚(:Ý üüâÔĸí % disk.nvme0n1.out‚20539.3134765625 disk.sda2.in‚0% disk./.usageGB‚243.18076705932617 disk.nvme0n1.in‚ 338.32421875 proc.cpu.threads‚17) disk./.usagePercent‚16.32006531960554 disk.sda2.out‚0 disk.rbd0.in‚ 0.0078125 disk.sda.in‚0 disk.rbd1.out‚ 9.205078125 disk.dm-0.in‚ 338.32421875 disk.sda.out‚ 1984.9609375$ memory_percent‚86.93704605102539' proc.memory.availableMB‚ 2140.234375 disk.sda3.out‚ 1984.9609375 disk.drbd2080.out‚0 disk.drbd2265.out‚0) proc.memory.percent‚17.21038818359375 network.sent‚ 4.365636e+06 network.recv‚681748" disk.nvme0c0n1.in‚ 338.32421875' disk.nvme0c0n1.out‚20539.3134765625 disk.drbd2080.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2265.in‚0 cpu‚24.728046632967484" disk.dm-0.out‚20539.3134765625 disk.sda3.in‚0 disk.rbd0.out‚ 0.2041015625 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.dm-1.out‚ 1984.9609375 proc.memory.rssMB‚2819.75 disk.rbd1.in‚0giŽý®‚(:’ ‹ýâÔ¯ä×  gpu.0.gpu‚28.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚126.468' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.293599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0_gMk¯‚(:à ‹ýâÔ³å— disk.sda.out‚ 2003.58984375$ memory_percent‚86.93714141845703 proc.memory.rssMB‚2819.75 disk.drbd2265.out‚0 disk.sda3.in‚0 disk.drbd2265.in‚0 disk.rbd1.out‚ 9.205078125 proc.cpu.threads‚17 network.sent‚ 4.452908e+06" disk.dm-0.out‚20539.5400390625 disk.rbd0.out‚ 0.2041015625' disk.nvme0c0n1.out‚20539.5400390625& proc.memory.availableMB‚ 2140.21875 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875 disk.drbd2266.in‚0 disk.dm-1.out‚ 2003.58984375 disk.rbd1.in‚0 disk.dm-1.in‚0 disk.sda3.out‚ 2003.58984375 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda1.in‚0% disk.nvme0n1.out‚20539.5400390625 network.recv‚687539* disk./.usagePercent‚16.320029478550747% disk./.usageGB‚243.18023300170898 disk.dm-0.in‚ 338.32421875 disk.drbd2266.out‚0) proc.memory.percent‚17.21038818359375 disk.sda.in‚0 disk.sda1.out‚0" disk.nvme0c0n1.in‚ 338.32421875 cpu‚24.735380645982026E9Œ}2°- _wandbruntime‚1932  _runtime‚1932Bj£)DZÁ & lr-AdamW/pg1‚9.226729847486942e-05 & lr-AdamW/pg2‚9.226729847486942e-05 trainer/global_step‚2349 % _timestamp‚1.7883951593783507e+09 _runtime‚1932.358148841 _step‚187Á âÌ²Æ & lr-AdamW/pg2‚9.226729847486942e-05 trainer/global_step‚2349 % _timestamp‚1.7883951593783507e+09 _runtime‚1932.358148841 _step‚187 & lr-AdamW/pg1‚9.226729847486942e-05»1N72³- _wandbruntime‚1932  _runtime‚1932†È¬Ø ´ÏO aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚130507409337344 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚75821373 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32693716 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217795584 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚132297079083008 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1706193466822 _step‚188 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19671849 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1789644841984 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1715263071744 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126964904156614 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚125258727729152 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚129015209763840 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚132296836383744 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32693714 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43147185 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43147185 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚129015226803200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126965081268290 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1706353539138 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32693716 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚130730653761024 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31058689 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚431 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19671852 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11386409 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚130730472835584 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚130507434241024 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31058258 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚75840901 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1715426957824 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32693714 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚428 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚129015226803200 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1715426957824 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚130730653761024 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚130730472835584 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1715263071744 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11386837 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚125258710689792 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623›5e²Ú'µÔO O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1789644841984 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1715263071744 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126964904156614 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚125258727729152 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚129015209763840 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚132296836383744 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 315496448 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32693714 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43147185 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43147185 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚129015226803200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126965081268290 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1706353539138 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32693716 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚130730653761024 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31058689 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚431 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19671852 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11386409 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚130730472835584 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚130507434241024 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31058258 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚75840901 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1715426957824 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32693714 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚428 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚129015226803200 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1715426957824 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚130730653761024 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚130730472835584 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1715263071744 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11386837 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚125258710689792 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚130507409337344 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚75821373 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32693716 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217795584 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚132297079083008 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1706193466822 _step‚188 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19671849 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚623¼}ÜÞú2¶- _wandbruntime‚1933  _runtime‚1933}‰²Ià&·ÚM E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚130561411166208 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31070915 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 % _timestamp‚1.7883951600612893e+09 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚130561436069888 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚125310628413440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚130784754443264 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32707210 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75872415 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32707212 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75852861 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚43165203 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 434110464 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚130784754443264 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚127017528839722 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 160072524 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 418504017 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚28 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚132351819173376 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1707060498806 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19679911 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43145651 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚130784573504512 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11391007 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚132351578584576 trainer/global_step‚2349 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1706900426282 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11391463 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1715973818368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚127017705951606 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32707210 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _step‚189 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43145651 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚130784573504512 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11391007 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚132351578584576 trainer/global_step‚2349 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1706900426282 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 434110464 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 8967819264 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11391463 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1715973818368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚623 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚127017705951606 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32707210 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 _step‚189 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31070915 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 % _timestamp‚1.7883951600612893e+09 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚130561436069888 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚125310628413440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚130784754443264 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32707210 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75872415 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32707212 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚459 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75852861½˜pSA2¹- _wandbruntime‚1933  _runtime‚1933ýªeðZºÔ 1loss/SRqqt2/detection_loss‚2.7429893598323916 1loss/FBt3/assignment_loss‚0.38446330989006583 _runtime‚1933.042576952 /loss/FRt4/assignment_loss‚4.553452560534844 /loss/FRt4/detection_loss‚2.6528020783876762 1loss/SRqqt4/detection_loss‚2.7494223743188577 0loss/FBt4/assignment_loss‚0.5441532311621943 /loss/FBt2/detection_loss‚0.9947001997510588 .loss/FBt4/detection_loss‚2.904621950217656 0loss/FRt1/assignment_loss‚3.6343239152824487 0loss/FRt2/assignment_loss‚3.3716501023458396 2loss/SRqqt1/assignment_loss‚1.7308786090114991 epoch‚3 1loss/SRqqt4/assignment_loss‚2.230731274260849 /loss/FRt2/detection_loss‚2.4866679470068305 .loss/FBt1/detection_loss‚2.844154159228007 .loss/FBt3/detection_loss‚2.646499923430383 &loss/total_loss‚53.598031227113104 /loss/FRt3/assignment_loss‚4.072383561766292 -loss/FRt1/detection_loss‚2.36176918561642 1loss/SRqqt3/detection_loss‚1.5069662987640717 % _timestamp‚1.7883951600629652e+09 1loss/SRqqt2/assignment_loss‚2.796766443950374 1loss/FBt1/assignment_loss‚0.44729780509240097 1loss/FBt2/assignment_loss‚0.23118390860402946 /loss/FRt3/detection_loss‚2.1719202090458696 1loss/SRqqt1/detection_loss‚1.5617510327975528 trainer/global_step‚2349 _step‚190 2loss/SRqqt3/assignment_loss‚1.9764817868154951-+6_»Ù /loss/FBt2/detection_loss‚0.9947001997510588 .loss/FBt4/detection_loss‚2.904621950217656 0loss/FRt1/assignment_loss‚3.6343239152824487 0loss/FRt2/assignment_loss‚3.3716501023458396 2loss/SRqqt1/assignment_loss‚1.7308786090114991 epoch‚3 1loss/SRqqt4/assignment_loss‚2.230731274260849 /loss/FRt2/detection_loss‚2.4866679470068305 .loss/FBt1/detection_loss‚2.844154159228007 .loss/FBt3/detection_loss‚2.646499923430383 &loss/total_loss‚53.598031227113104 /loss/FRt3/assignment_loss‚4.072383561766292 -loss/FRt1/detection_loss‚2.36176918561642 1loss/SRqqt3/detection_loss‚1.5069662987640717 % _timestamp‚1.7883951600629652e+09 1loss/SRqqt2/assignment_loss‚2.796766443950374 1loss/FBt1/assignment_loss‚0.44729780509240097 1loss/FBt2/assignment_loss‚0.23118390860402946 /loss/FRt3/detection_loss‚2.1719202090458696 1loss/SRqqt1/detection_loss‚1.5617510327975528 trainer/global_step‚2349 _step‚190 2loss/SRqqt3/assignment_loss‚1.9764817868154951 1loss/SRqqt2/detection_loss‚2.7429893598323916 1loss/FBt3/assignment_loss‚0.38446330989006583 _runtime‚1933.042576952 /loss/FRt4/assignment_loss‚4.553452560534844 /loss/FRt4/detection_loss‚2.6528020783876762 1loss/SRqqt4/detection_loss‚2.7494223743188577 0loss/FBt4/assignment_loss‚0.5441532311621943¾´)c ‘¼‚(:† šýâÔÏÿ»  gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚176.968' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.3936 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*˜Ãk½‚(:à šýâÔ‚åÊ  proc.memory.rssMB‚2819.75) proc.memory.percent‚17.21038818359375 network.sent‚ 4.458302e+06 disk.drbd2265.out‚0 disk.sda1.out‚0 disk.sda3.out‚ 2022.73828125 disk.dm-0.in‚ 338.32421875! disk.dm-0.out‚20539.986328125 disk.drbd2266.in‚0 disk.sda2.out‚0 disk.rbd0.in‚ 0.0078125% disk./.usageGB‚243.18023300170898$ disk.nvme0n1.out‚20539.986328125* disk./.usagePercent‚16.320029478550747 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875& disk.nvme0c0n1.out‚20539.986328125 disk.drbd1716.out‚0 disk.drbd1716.in‚0 network.recv‚691388 disk.sda.out‚ 2022.73828125 disk.sda1.in‚0 disk.rbd1.out‚ 9.361328125 disk.rbd0.out‚ 0.2041015625 disk.nvme0n1.in‚ 338.32421875 cpu‚24.742268310394966 disk.sda.in‚0 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 2022.73828125 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0$ memory_percent‚86.93735599517822 disk.drbd2266.out‚0 disk.sda2.in‚0) proc.memory.availableMB‚ 2140.18359375 proc.cpu.threads‚17§áÛø’¾‚(:‡ ©ýâÔø†°  gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚37.0 gpu.0.powerWatts‚179.923' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.9846 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0íÊ^f¿‚(:Û ©ýâÔí¬) disk./.usagePercent‚16.32002973455828& disk.nvme0c0n1.out‚20540.220703125 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.sda3.out‚ 2042.3828125) proc.memory.percent‚17.21038818359375! disk.dm-0.out‚20540.220703125$ disk.nvme0n1.out‚20540.220703125 disk.sda.in‚0 proc.cpu.threads‚17 disk.dm-1.out‚ 2042.3828125( proc.memory.availableMB‚ 2140.1484375 disk.sda2.in‚0 disk.drbd1716.in‚0 disk.sda1.in‚0 disk.dm-1.in‚0 network.sent‚ 4.544927e+06% disk./.usageGB‚243.18023681640625 disk.sda2.out‚0 disk.sda3.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.dm-0.in‚ 338.32421875 disk.sda.out‚ 2042.3828125 disk.drbd2266.out‚0 proc.memory.rssMB‚2819.75 cpu‚24.749143495856142 disk.drbd2080.in‚0 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0$ memory_percent‚86.93757057189941 network.recv‚696400 disk.rbd0.out‚ 0.2041015625 disk.rbd1.out‚ 9.447265625>¼‘À‚(:† ¸ýâÔÎد  gpu.0.gpu‚25.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚139.685' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.937 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0-ÇõhÁ‚(:Ý ¸ýâԿЈ& disk.nvme0c0n1.out‚20540.736328125 disk.drbd1716.out‚0$ disk.nvme0n1.out‚20540.736328125 disk.drbd2265.in‚0 disk.dm-1.out‚ 2063.48828125 disk.sda2.out‚0 proc.cpu.threads‚17 network.sent‚ 4.551628e+06% disk./.usageGB‚243.18029403686523 disk.drbd2266.out‚0 disk.sda3.in‚0 disk.sda2.in‚0 disk.drbd2080.out‚0$ memory_percent‚86.93757057189941 disk.sda1.in‚0 disk.sda3.out‚ 2063.48828125* disk./.usagePercent‚16.320033574671296 disk.drbd2265.out‚0 disk.sda1.out‚0 disk.drbd1716.in‚0( proc.memory.availableMB‚ 2140.1484375 network.recv‚701766 disk.drbd2266.in‚0 disk.rbd1.in‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd2080.in‚0 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875 disk.sda.in‚0 disk.dm-1.in‚0 disk.dm-0.in‚ 338.32421875! disk.dm-0.out‚20540.736328125 disk.rbd0.out‚ 0.2041015625 disk.sda.out‚ 2063.48828125 proc.memory.rssMB‚2819.75) proc.memory.percent‚17.21038818359375 cpu‚24.7561596210648 disk.rbd1.out‚ 9.509765625̮ŗ2Â- _wandbruntime‚1970  _runtime‚1970síhÆÃÀ & lr-AdamW/pg1‚9.078286510707376e-05 & lr-AdamW/pg2‚9.078286510707376e-05 trainer/global_step‚2399 $ _timestamp‚1.788395197738163e+09 _runtime‚1970.717938274 _step‚191¶eàøËÄÅ & lr-AdamW/pg1‚9.078286510707376e-05 & lr-AdamW/pg2‚9.078286510707376e-05 trainer/global_step‚2399 $ _timestamp‚1.788395197738163e+09 _runtime‚1970.717938274 _step‚191¿7o¯2Å- _wandbruntime‚1970  _runtime‚1970™Ñ5îÖ'ÆÐO WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11620523 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚135034324983296 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚127854596872192 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33368516 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚131684705879040 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚133435715035648 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1750992117248 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1750828231168 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31695322 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚133207525684224 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚127854613911552 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20074802 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1741728684146 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 _runtime‚1970.719344839 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚133435534110208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1826799299072 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44030611 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚77418653 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚129596342595698 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚129596165484022 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1741568611830 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883951977392952e+09 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1826583600640 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44050137 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚133435715035648 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1750992117248 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚131684705879040 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44050137 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33368516 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33368514 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚135034084381184 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44030611 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚192 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77399125 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33368514 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚131684722918400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 trainer/global_step‚2399 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31695787 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚133435534110208 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809Èæ\¨ü&ÇÕO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33368516 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33368514 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚135034084381184 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚623 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44030611 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215698432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚192 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77399125 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33368514 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚131684722918400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚416 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚207 trainer/global_step‚2399 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 8967819264 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31695787 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 434110464 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚133435534110208 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242753536 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11620523 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚135034324983296 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚127854596872192 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8675334144 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33368516 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚131684705879040 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚133435715035648 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1750992117248 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1750828231168 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31695322 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚28 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚133207525684224 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚127854613911552 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20074802 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 422324736 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 180925440 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 163886080 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1741728684146 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9141485568 _runtime‚1970.719344839 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚133435534110208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1826799299072 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44030611 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23488 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23765 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚77418653 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚623 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 240602112 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚129596342595698 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚129596165484022 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1741568611830 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9575596032 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 % _timestamp‚1.7883951977392952e+09 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1826583600640 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44050137 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚133435715035648 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1750992117248 \HDeviceStatsMonitor.on_trŠ%ßain_batch_start/allocated_bytes.large_pool.freed‚131684705879040 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44050137 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0Àeƒá2È- _wandbruntime‚1971  _runtime‚1971«ë5¡ß&ÉÙM XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚135088825817600 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9141485568 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23765 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33382012 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33382010 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 _runtime‚1971.398147297 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44068155 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20082861 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 trainer/global_step‚2399 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚133261502609408 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31707962 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚77450167 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33382010 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚133489815717888 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚127906514595840 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚77430613 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚133261527513088 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚44048603 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚133489634779136 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20082858 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44068155 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1742275571290 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1751702877184 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 $ _timestamp‚1.788395198417863e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11625104 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77450167 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚129648790167130 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚129648967279014 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1742435643814 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1751538977792 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11625573 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚472 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 219879424 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77430613 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33382012 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚131738095801344 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚131738112840704 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44048603 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1827543087616 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33382010 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚207 _runtime‚1971.398147297 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44068155 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 163899392 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20082861 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚416 trainer/global_step‚2399 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚133261502609408 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9141485568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31707962 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚77450167 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33382010 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚133489815717888 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚127906514595840 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚77430613 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚133261527513088 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19552 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚44048603 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚133489634779136 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20082858 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44068155 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1742275571290 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1751702877184 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚979 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚416 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 $ _timestamp‚1.788395198417863e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11625104 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 145514496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242753536 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8675334144 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚623 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77450167 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚129648790167130 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9575596032 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 315496448 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚129648967279014 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1742435643814 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9575596032 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1751538977792 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11625573 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19554 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚472 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244783104 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 219879424 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 163899392 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 434110464 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9352659968 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77430613 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33382012 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 180938752 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚131738095801344 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚131738112840704 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 8967819264 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44048603 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1827543087616 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚207 B&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8"Purity/0FRt*SRqqt2FBt/event_purity‚0.306743860244751 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.991277277469635 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.15962596237659454 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9717262387275696 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 8"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.412202388048172 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.11034481972455978 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13705262541770935 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 jet/accuracy_5_of_6‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48122531175613403 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.25593915581703186 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3020145297050476 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.21605342626571655 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9605262875556946 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.29801586270332336 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.40766915678977966 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4023268222808838 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 +jet/accuracy_2_of_2‚0.08605142682790756 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.37304750084877014 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37137678265571594 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3188580274581909 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08473546802997589 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3895929157733917 *jet/accuracy_1_of_1‚0.3961126208305359 particle/accuracy_3_of_3‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9135803580284119 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2905261218547821 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.29487180709838867 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_2_of_7‚1 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35775983333587646 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.051394976675510406 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03879557177424431 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.32618141174316406 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.18955454230308533 ("Purity/*FRt4SRqqt*AØ%FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0694444477558136 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3993055522441864 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.05784512311220169 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 jet/accuracy_7_of_7‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9859739542007446 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1098843663930893 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994135499000549 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.969005286693573 particle/accuracy_2_of_6‚1 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.198222815990448 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3738205134868622 6 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.143036350607872 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.024259259924292564 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.34464284777641296 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 -jet/accuracy_4_of_4‚0.0021601382177323103 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.009105813689529896 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.32581764459609985 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10794380307197571 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9978674650192261 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9845661520957947 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1666666716337204 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.990491509437561 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08473895490169525 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9886623620986938 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07055390626192093 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6503980159759521 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.16308021545410156 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5167468190193176 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.20066668093204498 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.38672110438346863 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04169226810336113 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.35866934061050415 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_2_of_2‚1 ,jet/accuracy_4_of_6‚0.017543859779834747 particle/accuracy_4_of_6‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17667575180530548 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.24436861276626587 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_5_of_5‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2477855086326599 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8090512752532959 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15359166264533997 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.653508722782135 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006114075426012278 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 particle/accuracy_1_of_7‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.19333332777023315 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.46666669845581055 ,jet/accuracy_3_of_4‚0.041975270956754684 +particle/specificity‚0.9480608701705933 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.24074074625968933 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.08690754324197769 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3070175349712372 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8968254327774048 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19432732462882996 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.15843620896339417 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08460228145122528 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.009667987003922462 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3762957751750946 particle/accuracy_1_of_1‚1 particle/accuracy_1_of_5‚1 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.12821675837039948 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 *jet/accuracy_2_of_4‚0.2810114622116089 6validation_average_jet_accuracy‚0.3646549880504608 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9899402260780334 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9958208203315735 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.45787033438682556 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9899451732635498 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666150093079 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 % _timestamp‚1.7883952468539221e+09 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 particle/accuracy_6_of_7‚1 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.32858356833457947 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.27444836497306824 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.08796296268701553 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 _step‚195 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 jet/accuracy_3_of_7‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33536532521247864 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.42460107803344727 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.19098889827728271 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.17304886877536774 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4924319386482239 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4292680621147156 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.40123459696769714 (particle/accuracy‚0.8395392894744873 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2653520107269287 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.12482284009456635 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3288889229297638 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.12602296471595764 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16398997604846954 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 (particle/f_score‚0.33013054728507996 )loss/val_total_loss‚54.60294152421758 9"Purity/0FRt1SRqqt2FBt/event_purity‚0.0591137558221817 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.10986046493053436 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 particle/accuracy_1_of_3‚1 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_4_of_7‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.33044689893722534 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 particle/accuracy_7_of_7‚0.5 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9821928143501282 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.09375 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 -jet/accuracy_4_of_5‚0.0026881720405071974 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.4117647111415863 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 +jet/accuracy_3_of_5‚0.05796130374073982 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 epoch‚3 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3414372205734253 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3202982246875763 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.4888888895511627 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7669857740402222 particle/accuracy_1_of_2‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1107206791639328 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.0714285746216774 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3493172228336334 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3493172228336334 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 *jet/accuracy_1_of_6‚0.8947368264198303 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6428571343421936 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.27775052189826965 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3377481997013092 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.3464912474155426 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.11958875507116318 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10701409727334976 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974358439445496 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.18060196936130524 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3406791388988495 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4078729450702667 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3427404761314392 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0694444477558136 particle/accuracy_5_of_6‚1 jet/accuracy_5_of_7‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3827537000179291 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.28913962841033936 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_2_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.029902715235948563 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.1593085378408432 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 trainer/global_step‚2415 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8293997049331665 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.48165643215179443 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_3‚0.01727767288684845 particle/accuracy_1_of_6‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08703085035085678 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25992652773857117 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42674097418785095 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08460228145122528 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_4‚1 +jet/accuracy_2_of_6‚0.35087719559669495 +jet/accuracy_3_of_6‚0.12280702590942383 jet/accuracy_4_of_7‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9907498955726624 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9927814602851868 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9965999126434326 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9842800498008728 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010052557103335857 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3919890522956848 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14074215292930603 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.2874622046947479 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.44771096110343933 particle/accuracy_3_of_4‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.13806964457035065 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3186059892177582 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4736558794975281 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12320870906114578 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9944690465927124 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9701745510101318 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9928139448165894 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.43122345209121704 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 _runtime‚2019.835925496 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.373535692691803 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.036834072321653366 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3122035264968872 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.003315536305308342 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.33175578713417053 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.45555028319358826 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1518523395061493 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.2532268762588501 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2142857164144516 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_5‚1 jet/accuracy_6_of_6‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.34814468026161194 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.919540286064148 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2907901406288147 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.1651785671710968 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 7"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.40476194024086 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.26889243721961975 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.018049156293272972 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.10967741906642914 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0023822563234716654 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.1875 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.28864800930023193 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.09894656389951706 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2713288366794586 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.04248365759849548 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16106931865215302 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15661877393722534 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.34674331545829773 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4328550100326538 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2586405575275421 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8960407376289368 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.1875 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.331023246049881 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6325610280036926 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4787139594554901 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2708333432674408 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2537878751754761 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2628116309642792 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9903733134269714 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.967129647731781 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16296972334384918 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11493798345327377 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08749999850988388 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_2_of_5‚0.3268562853336334 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2601550221443176 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.40476194024086 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.38749998807907104 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879914522171 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.234375 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013973614200949669 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4886375367641449 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1071428582072258 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03483634069561958 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1213955283164978 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6359648704528809 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚1 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10504235327243805 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0714285746216774 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.28592410683631897 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15284396708011627 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4174443483352661 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34294217824935913 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_2‚0.6504958868026733 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3461538553237915 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.10000000149011612 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.26490724086761475 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03253960236907005 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.10733334720134735 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214‡—økÍÖÎ¥ ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646jêçù particle/accuracy_2_of_2‚1 ,jet/accuracy_4_of_6‚0.017543859779834747 particle/accuracy_4_of_6‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.17667575180530548 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.24436861276626587 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 particle/accuracy_5_of_5‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2477855086326599 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8090512752532959 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15359166264533997 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 8"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.653508722782135 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006114075426012278 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 particle/accuracy_1_of_7‚1 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.19333332777023315 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.46666669845581055 ,jet/accuracy_3_of_4‚0.041975270956754684 +particle/specificity‚0.9480608701705933 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.24074074625968933 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.08690754324197769 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3070175349712372 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8968254327774048 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19432732462882996 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.15843620896339417 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08460228145122528 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.009667987003922462 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3762957751750946 particle/accuracy_1_of_1‚1 particle/accuracy_1_of_5‚1 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.12821675837039948 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 *jet/accuracy_2_of_4‚0.2810114622116089 6validation_average_jet_accuracy‚0.3646549880504608 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9899402260780334 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9958208203315735 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.45787033438682556 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9899451732635498 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666150093079 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 % _timestamp‚1.7883952468539221e+09 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 particle/accuracy_6_of_7‚1 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.32858356833457947 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.27444836497306824 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.08796296268701553 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 _step‚195 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 jet/accuracy_3_of_7‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33536532521247864 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.42460107803344727 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.40625 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.19098889827728271 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.17304886877536774 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4924319386482239 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4292680621147156 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.40123459696769714 (particle/accuracy‚0.8395392894744873 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2653520107269287 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.12482284009456635 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3288889229297638 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.12602296471595764 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16398997604846954 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 (particle/f_score‚0.33013054728507996 )loss/val_total_loss‚54.60294152421758 9"Purity/0FRt1SRqqt2FBt/event_purity‚0.0591137558221817 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.10986046493053436 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 particle/accuracy_1_of_3‚1 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_4_of_7‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.33044689893722534 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 particle/accuracy_7_of_7‚0.5 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4333333373069763 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9821928143501282 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.09375 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 -jet/accuracy_4_of_5‚0.0026881720405071974 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.4117647111415863 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 +jet/accuracy_3_of_5‚0.05796130374073982 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 epoch‚3 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3414372205734253 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3202982246875763 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.4888888895511627 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7669857740402222 particle/accuracy_1_of_2‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1107206791639328 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 particle/accuracy_3_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.0714285746216774 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3493172228336334 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3493172228336334 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 *jet/accuracy_1_of_6‚0.8947368264198303 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6428571343421936 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.27775052189826965 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.3377481997013092 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.3464912474155426 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.11958875507116318 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10701409727334976 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974358439445496 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.18060196936130524 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3406791388988495 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4078729450702667 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3427404761314392 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0694444477558136 particle/accuracy_5_of_6‚1 jet/accuracy_5_of_7‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3827537000179291 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.28913962841033936 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_2_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.029902715235948563 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.1593085378408432 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 trainer/global_step‚2415 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8293997049331665 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.48165643215179443 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 +jet/accuracy_3_of_3‚0.01727767288684845 particle/accuracy_1_of_6‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08703085035085678 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25992652773857117 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.42674097418785095 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08460228145122528 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_4‚1 +jet/accuracy_2_of_6‚0.35087719559669495 +jet/accuracy_3_of_6‚0.12280702590942383 jet/accuracy_4_of_7‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9907498955726624 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9927814602851868 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9965999126434326 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9842800498008728 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010052557103335857 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3919890522956848 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14074215292930603 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.2874622046947479 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.44771096110343933 particle/accuracy_3_of_4‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.13806964457035065 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3186059892177582 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4736558794975281 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12320870906114578 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9944690465927124 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9701745510101318 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9928139448165894 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.43122345209121704 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 _runtime‚2019.835925496 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.373535692691803 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.036834072321653366 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3122035264968872 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.003315536305308342 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.33175578713417053 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.45555028319358826 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1518523395061493 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 +particle/sensitivity‚0.2532268762588501 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.2142857164144516 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_6_of_6‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_5‚1 jet/accuracy_6_of_6‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.34814468026161194 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.919540286064148 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2907901406288147 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.1651785671710968 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 7"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.40476194024086 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.26889243721961975 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.018049156293272972 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.10967741906642914 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0023822563234716654 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.1875 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.28864800930023193 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.09894656389951706 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2713288366794586 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.04248365759849548 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16106931865215302 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.15661877393722534 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.34674331545829773 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.05000000074505806 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4328550100326538 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2586405575275421 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.8960407376289368 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.1875 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.331023246049881 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6325610280036926 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4787139594554901 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2708333432674408 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2537878751754761 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3461538553237915 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3333333432674408 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2628116309642792 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9903733134269714 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.967129647731781 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16296972334384918 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11493798345327377 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08749999850988388 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_2_of_5‚0.3268562853336334 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2601550221443176 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.40476194024086 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.38749998807907104 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.08574879914522171 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.234375 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013973614200949669 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4886375367641449 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1071428582072258 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03483634069561958 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.1213955283164978 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6359648704528809 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚1 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10504235327243805 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0714285746216774 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.28592410683631897 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.15284396708011627 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4174443483352661 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34294217824935913 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_2‚0.6504958868026733 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3461538553237915 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.10000000149011612 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.26490724086761475 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03253960236907005 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.10733334720134735 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.11783500760793686 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.32231608033180237 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.015290549956262112 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11404642462730408 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3100760579109192 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13137105107307434 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9987593293190002 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8"Purity/0FRt*SRqqt2FBt/event_purity‚0.306743860244751 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.991277277469635 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.15962596237659454 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9717262387275696 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 8"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.412202388048172 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.11034481972455978 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13705262541770935 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 jet/accuracy_5_of_6‚0 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.48122531175613403 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.25593915581703186 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3020145297050476 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.21605342626571655 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9605262875556946 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.29801586270332336 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.4285714328289032 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.40766915678977966 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4023268222808838 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 +jet/accuracy_2_of_2‚0.08605142682790756 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.37304750084877014 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37137678265571594 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3188580274581909 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.08473546802997589 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3895929157733917 *jet/accuracy_1_of_1‚0.3961126208305359 particle/accuracy_3_of_3‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9135803580284119 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2905261218547821 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.29487180709838867 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_2_of_7‚1 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35775983333587646 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.051394976675510406 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03879557177424431 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.32618141174316406 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.18955454230308533 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0694444477558136 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ,&Purity/*FRt3SRqqt3FBt/event_pråneoportion‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3993055522441864 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.05784512311220169 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 jet/accuracy_7_of_7‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9859739542007446 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1098843663930893 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994135499000549 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.969005286693573 particle/accuracy_2_of_6‚1 8"Purity/*FRt*SRqqt*FBt/event_purity‚0.198222815990448 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3738205134868622 6 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.143036350607872 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.024259259924292564 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.34464284777641296 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 -jet/accuracy_4_of_4‚0.0021601382177323103 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.009105813689529896 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.32581764459609985 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10794380307197571 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9978674650192261 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9845661520957947 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1666666716337204 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 6 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.990491509437561 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08473895490169525 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9886623620986938 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.07055390626192093 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6503980159759521 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.16308021545410156 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5167468190193176 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.20066668093204498 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.38672110438346863 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.04169226810336113 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.35866934061050415Ã~۵đׂ(:† ôýâÔÿè­  gpu.0.gpu‚21.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚35.0 gpu.0.powerWatts‚175.085' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.017 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“]0ÝfØ‚(:Û ôýâÔíŽÍ network.recv‚719697" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.rbd1.out‚ 9.6982421875 disk.dm-0.in‚ 338.32421875" disk.dm-0.out‚20661.6923828125 disk.sda2.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚ 338.32421875 disk.rbd0.out‚ 0.2041015625 cpu‚24.8977410076013 disk.sda3.out‚ 2142.3125 disk.drbd2080.out‚0 disk.drbd2266.out‚0$ memory_percent‚87.46781349182129 proc.cpu.threads‚17 network.sent‚ 4.655259e+06% disk./.usageGB‚243.18152236938477 disk.drbd1716.in‚0 disk.sda.out‚ 2142.3125( proc.memory.availableMB‚ 2053.2734375# proc.memory.rssMB‚ 2905.82421875 disk.drbd1716.out‚0% disk.nvme0n1.out‚20661.6923828125 disk.drbd2265.in‚0 disk.dm-1.out‚ 2142.3125* proc.memory.percent‚17.735743522644043 disk.sda3.in‚0' disk.nvme0c0n1.out‚20661.6923828125 disk.drbd2080.in‚0 disk.sda1.in‚0 disk.sda1.out‚0) disk./.usagePercent‚16.32011600909732 disk.rbd1.in‚0 disk.rbd0.in‚ 0.0078125 disk.sda.in‚0·¨Ä’Ù‚(:‡ ƒþâÔÅ·Þ  gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚11.784744262695312* gpu.0.memoryAllocatedBytes‚ 10123018240 gpu.0.temp‚36.0 gpu.0.powerWatts‚175.614' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.1228 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¬V‡·tÚ‚(:é ƒþâÔ†ý©  disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-0.in‚ 338.32421875 cpu‚24.903306917298003 disk.drbd2265.in‚0 disk.drbd2080.out‚0% disk./.usageGB‚243.18146514892578 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0% disk.nvme0n1.out‚20661.9189453125 disk.sda.out‚ 2158.71484375# proc.memory.rssMB‚ 2905.83203125* proc.memory.percent‚17.735791206359863 network.sent‚ 4.758228e+06 network.recv‚726154) disk./.usagePercent‚16.32011216898431 disk.drbd2266.in‚0 disk.dm-1.out‚ 2158.71484375 disk.sda2.out‚0( proc.memory.availableMB‚ 2053.3984375 disk.sda3.in‚0 disk.rbd1.in‚0 disk.sda2.in‚0 disk.rbd0.in‚ 0.0078125 disk.nvme0n1.in‚ 338.32421875 proc.cpu.threads‚17 disk.drbd2266.out‚0" disk.nvme0c0n1.in‚ 338.32421875 disk.drbd1716.in‚0 disk.rbd0.out‚ 0.2041015625 disk.drbd2265.out‚0 disk.sda3.out‚ 2158.71484375 disk.rbd1.out‚ 9.6982421875' disk.nvme0c0n1.out‚20661.9189453125 disk.sda.in‚0" disk.dm-0.out‚20661.9189453125$ memory_percent‚87.467050552368160=<Ë2Û- _wandbruntime‚2047  _runtime‚2047xÌ"PÇÜÁ _step‚196 & lr-AdamW/pg1‚8.927288172159174e-05 & lr-AdamW/pg2‚8.927288172159174e-05 trainer/global_step‚2449 % _timestamp‚1.7883952747747803e+09 _runtime‚2047.754598697®¸e—ÌÝÆ & lr-AdamW/pg1‚8.927288172159174e-05 & lr-AdamW/pg2‚8.927288172159174e-05 trainer/global_step‚2449 % _timestamp‚1.7883952747747803e+09 _runtime‚2047.754598697 _step‚196Ĭ¤—2Þ- _wandbruntime‚2047  _runtime‚2047tàÂÕ'ßÏO _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1878446427648 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1791402360736 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11974324 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚131528640231424 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34366894 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1801062199296 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 213601280 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1800898313216 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11974817 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚133320202664476 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚138980932249088 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚138980693744128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚135530920457216 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚133320025552800 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 $ _timestamp‚1.788395274775859e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1801062199296 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 238504960 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚137331818770432 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚135530937496576 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _runtime‚2047.759278388 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 _step‚197 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚79531610 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1800898313216 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79551138 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚45164718 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1878660028928 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20720668 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416|TÓžÚ'àÔO IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚416 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1878446427648 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1791402360736 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9575596032 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9352659968 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11974324 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚131528640231424 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚207 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34366894 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 8967819264 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1801062199296 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚979 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 213601280 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 434110464 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1800898313216 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23488 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11974817 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚133320202664476 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 180925440 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 422324736 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚138980932249088 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚138980693744128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚135530920457216 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚133320025552800 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 $ _timestamp‚1.788395274775859e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1801062199296 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2513 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚960 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 238504960 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 177111676 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 418504017 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚137331818770432 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚135530937496576 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19528 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 145514496 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9141485568 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚416 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9352659968 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚623 _runtime‚2047.759278388 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19526 _step‚197 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚79531610 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23765 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9056820809 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚207 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1800898313216 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79551138 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚45164718 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1878660028928 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19526 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 160072316 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9141485568 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19528 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2513 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 163886080 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20720668 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 434110464Å?‹12á- _wandbruntime‚2048  _runtime‚2048ß8_hÁâÚM PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 422324736 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23765 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚454 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9575596032 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79582652 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19552 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23488 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1801609059840 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚135584310379520 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚960 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚133372650235908 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34380388 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2513 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚623 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79582652 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚139035676290048 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1792269392720 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚45202262 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚45182710 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1792109320196 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚416 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9056820809 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚207 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚623 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 215685120 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9141485568 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1801772959232 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11979108 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚34380390 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚45182710 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9352659968 WCDeviceStatsMonitor.on_train_batch_en